Skip to content

Administration overview

In this guide we will refer to one of the example channel delivered as part of the standalone punchplatform. If you do not have that at hand no worry, this is not necessary to understand what follows. Should you have one, each chapter will be associated to one or several example commands you can run on your standlone platform.

The punchplatform provides its users with very few concepts to understand and a minimal set of commands. Each user

Configuration Management

Tenant, Channels and Services

Every data processing pipeline, every administrative command is defined as part of a tenant. The configuration folder layout (in zookeeper or in your local filesystem) has a REST-like tree organisation starting with the tenant name, then have subfolders for channels and services.

Channels contain your data processing logic. A channel is a data processing pipeline, possibly made up from several pieces : storm topologies, spark jobs, shiva tasks etc.. You act upon a channel using single commands : start, stop, reload.

Services contain your tenant administrative tasks. In there you setup administrative tasks such as data moving or deletion. Services support only two commands : start and stop. More precisely:

Channels and Services status

The runtime status of channels or services is one of the following.

 State  Meaning
ACTIVE  the channel or service is running in nominal condition.
STOPPED  the channel or service is stopped.
PARTIAL  this is a non nominal situation. One of your channel or service sub task is not running anymore, most probably because an operator stopped it using an external or cli command

Administration Tools

The punchplatform exposes a minimal set of commands to users. This is actually good news, it cannot be simpler to operate. The basic operations are to start stop channels or services. Channels support one additional operation : reload. These commands are exposed in two ways: from a command line terminal, or from the punch Kibana UI.

Administration Console

The platform superuser(s) are provided with local unix accounts and terminal commands. These are essentially the punchplatform-channel.sh and punchplatform-service.sh commands.

The various configuration files can be edited using standard file editors. However to actually apply the changes, their content must be pushed to zookeeper. That is, the latest applied platform and tenant configuration is always stored in zookeeper.

image

Reference configuration in zookeeper and relation to (optional) git repository

Because the operator environment is distributed (multiple operators, multiple machines), the reference configuration is stored in the administration zookeeper cluster, and must be retrieved using punchplatform-getconf.sh before doing changes to the platform configuration, and saved using 'punchplatform-putconf.sh` after changes, so that other operators can access it later (and for example see the up to date version in the punchplatform kibana plugin).

Note that it is ALSO a very-much-advised-good-practice to maintain a git configuration management of your punchplatform configuration tree, for traceability/rollbacks purpose in production environments. You should then alway commit and push the updated reference configuration to your central git repository for configuration management. BUT this does NOT make git the 'official' reference for the applicable configuration of your platform : the reference version is the one that has been stored using 'punchplatform-putconf.sh', so always work using this reference version, even if you maintain a git repository !

Administration Kibana GUI

Regular per tenant users are provided with a secured GUI. There, they can act only on their tenant configurations. The Kibana Gui let authorized users stop/start or reload channels and services.

Administration REST API

The punchplatform exposes some resources as REST APIs, in particular the health status of the platform. This makes it very simple to monitor a punchplatform from an external monitoring tool.