Skip to content

Operator

Abstract

the operator is a punch environment that provides a terminal user with all the necessary command.

Overview

Refer to the operator documentation documentation for a detailed description.

When deploying a punch clustered platform, you typically identify one or several servers to provide the platform operator with the necessary administration environment. The punch deployer takes care of installing all the required command, environment variables and binaries automatically. In the rest of this chapter we descrobe the operator nodes layout.

Operator configuration

The various operator commands are provided in a dedicated folder, typically under /opt or /data/opt root. Here is its layout

punch-operator-6.1.0/
├── bin
│   ├── archives-housekeeping
│   ├── channelctl
│   ├── channels-monitoring
│   ├── commons-lib.sh
│   ├── elastalert
│   ├── elasticsearch-housekeeping
│   ├── hjson-manager
│   ├── logstash
│   ├── mlflow_api
│   ├── planctl
│   ├── platform-monitoring
│   ├── punchctl
│   ├── punchextractor
│   ├── punchlinectl
│   ├── punchplatform-analytics-scan.sh
│   ├── punchplatform-env.sh
│   ├── punchplatform-getconf.sh
│   ├── punchplatform-kafka-consumers.sh
│   ├── punchplatform-kafka-topics.sh
│   ├── punchplatform-lib.sh
│   ├── punchplatform-log-injector.sh
│   ├── punchplatform-puncher.sh
│   ├── punchplatform-push-es-templates.sh
│   ├── punchplatform-pyspark-scan.sh
│   ├── punchplatform-setup-kibana.sh
│   ├── punchplatform-version.sh
│   ├── punchplatform-zookeeper-console.sh
│   ├── renderJinja2Template.py
│   ├── resourcectl
│   ├── sparkctl
│   └── storectl
├── log4j2
│   ├── log4j2-channel.xml
│   ├── log4j2.properties
│   ├── log4j2-sparkctl.properties
│   ├── log4j2-sparkctl.xml
│   └── log4j2-stdout.xml
├── punchplatform.properties
└── resolv.hjson

In addition the the various operator commands, this folder also contains logger configuration files, and and two punch platform level configuration files resolv.hjson and punchplatform.properties generated at deployment time.

Operator binaries

The binaries and libraries (i.e. executable) files are provided in a separate folder. These are provided by a punch uber binary package named punchplatform-binaries-*. The punch deployer install it on each punch node. Here is its layout:

punchplatform-binaries-6.1.0/
├── apache-storm-2.1.0
├── lib
│   ├── spark/
│   ├── pyspark/
│   ├── storm/
│   ├── punchplatform-command-cli-6.1.0-jar-with-dependencies.jar
│   ├── punchplatform-gateway-app-6.1.0.jar
│   ├── punchplatform-log-injector-6.1.0-jar-with-dependencies.jar
│   ├── punchplatform-monitoring-service-6.1.0-jar-with-dependencies.jar
│   ├── punchplatform-puncher-6.1.0-jar-with-dependencies.jar
│   ├── punchplatform-shiva-app-6.1.0-jar-with-dependencies.jar
│   └── ...
├── logstash-7.4.2
├── resources
└── spark-2.4.3-bin-hadoop2.7

For example, punchlinectl command uses punchplatform-command-cli-*-jar-with-dependencies.jar while punchplatform-log-injector.sh uses punchplatform-log-injector-*-jar-with-dependencies.jar

Patch an Operator command

Refer to this documentation