Operator

Abstract

the operator is a punch environment that provides a terminal user with all the necessary command.

Overview

When deploying a clustered punch platform, you typically identify one or several servers to provide the platform operator with the necessary administration environment. The punch deployer takes care of installing all the required commands, environment variables and binaries automatically. In the rest of this chapter we describe the operator nodes layout.

Operator configuration

The various operator commands are provided in a dedicated folder, typically under /opt or /data/opt root. Here is its layout

punch-operator-6.1.0/
├── bin
│   ├── archives-housekeeping
│   ├── channelctl
│   ├── channels-monitoring
│   ├── commons-lib.sh
│   ├── elastalert
│   ├── elasticsearch-housekeeping
│   ├── hjson-manager
│   ├── logstash
│   ├── mlflow_api
│   ├── planctl
│   ├── platform-monitoring
│   ├── punchctl
│   ├── punchextractor
│   ├── punchlinectl
│   ├── punchplatform-analytics-scan.sh
│   ├── punchplatform-env.sh
│   ├── punchplatform-getconf.sh
│   ├── punchplatform-kafka-consumers.sh
│   ├── punchplatform-kafka-topics.sh
│   ├── punchplatform-lib.sh
│   ├── punchplatform-log-injector.sh
│   ├── punchplatform-puncher.sh
│   ├── punchplatform-push-es-templates.sh
│   ├── punchplatform-pyspark-scan.sh
│   ├── punchplatform-setup-kibana.sh
│   ├── punchplatform-version.sh
│   ├── punchplatform-zookeeper-console.sh
│   ├── renderJinja2Template.py
│   ├── resourcectl
│   ├── sparkctl
├── log4j2
│   ├── log4j2-channel.xml
│   ├── log4j2.properties
│   ├── log4j2-sparkctl.properties
│   ├── log4j2-sparkctl.xml
│   └── log4j2-stdout.xml
├── punchplatform.properties
└── resolv.hjson

In addition to the various operator commands, this folder also contains the logger configuration files, and two platform level configuration files resolv.hjson and punchplatform.properties generated at deployment time.

Operator binaries

The binaries and libraries (i.e. executable) files are provided in a separate folder. These are provided by a punch uber binary package named punchplatform-binaries-*. The punch deployer install it on each punch node. Here is its layout:

punchplatform-binaries-6.1.0/
├── apache-storm-2.1.0
├── lib
│   ├── spark/
│   ├── pyspark/
│   ├── storm/
│   ├── punchplatform-command-cli-6.1.0-jar-with-dependencies.jar
│   ├── punchplatform-gateway-app-6.1.0.jar
│   ├── punchplatform-log-injector-6.1.0-jar-with-dependencies.jar
│   ├── punchplatform-monitoring-service-6.1.0-jar-with-dependencies.jar
│   ├── punchplatform-puncher-6.1.0-jar-with-dependencies.jar
│   ├── punchplatform-shiva-app-6.1.0-jar-with-dependencies.jar
│   └── ...
├── logstash-7.4.2
├── resources
└── spark-2.4.3-bin-hadoop2.7

For example, punchlinectl command uses punchplatform-command-cli-*-jar-with-dependencies.jar while punchplatform-log-injector.sh uses punchplatform-log-injector-*-jar-with-dependencies.jar

Patch an Operator command

Refer to this documentation