HOWTO shutdown a punchplatform

Why do that¶

You anticipate a power grid shutdown or have planned a heavy maintenance You want everything to start nicely and resilient again afterwards

Prerequisites¶

You will need a sudoer account working on all PunchPlatform servers. It is easier if you have access to a PunchPlatform deployment server

Warning

It is your responsibility to ensure that no more events/logs are sent to your PunchPlatform input, otherwise you may cause data loss in the networking layers (UDP, TCP system buffers or in-transit packets).

What to do¶

Overview¶

PunchPlatform is resilient. To avoid resilience mechanism to kick in during shutdown or restart, you will need to decide if you want your processing (PunchPlatform channels<Channel>).

If you want to prevent processing to restart when PunchPlatform will be restarted, you have to shutdown the Storm topologies before shutting down other PunchPlatform services.

Multiple PunchPlatform services depend on zookeeper for their critical data storage (cluster management, recovery data check points). For this reason, zookeeper must always be stopped last, and started first. The PunchPlatform components using zookeeper are : Kafka, Storm Clusters component, Storm topologies (that can occasionally be used outside of a Storm cluster), PunchPlatform admin.

Because storm topologies are using Kafka and Elasticsearch, they should be stopped before those dependencies, and started last.

All following steps shell commands must be invoked while connected as a system administrator on the ansible deployment machine, with PATH and PUNCHPLATFORM_CONF_DIR variable initialized (usually through the admin user .bashrc file) to point to the bin directory of the PunchPlatform deployer, and to the root of the platform configuration local git repository.

Shutdown Storm topologies¶

Warning

This should be skipped if the desired behaviour upon PunchPlatform future restart is to automatically run the same channels topologies as are currently running.

Access the PunchPlatform admin service, and consult the \'Channels\' tab to determine all the tenants that have channels listed.
Connected through ssh to a PunchPlatform administration server as a PunchPlatform operator (e.g. the ppadmin sample account deployed by PunchPlatform deployer by default), run for each \<tenantName> identified at the previous step :
```
channelctl stop
```

Shutdown Zookeeper cluster(s)¶

Ensure your local git repository is up to date as compared to platform reference bare repository. ( Your reference repository is indicated in the punchplatform_conf_repo_git_url setting of your punchplatform.properties file located in your $PUNCHPLATFORM_CONF_DIR).
```
git pull ssh://adm-infra@ppadmin/mnt/git/soc_ref
```
Stop zookeeper services (by requesting supervisord service)
```
punchplatform-deployer.sh --ssh zookeeper_servers 'supervisorctl stop zookeeper' -kK
```
(If your remote system admin account is not the same as your local login, add [-u ])

Shut down Storm cluster(s)¶

punchplatform-deployer.sh --ssh storm_servers 'supervisorctl stop all' -kK

(If your remote system admin account is not the same as your local login, add [-u ])

Shut down all remaining applicative components of PunchPlatform¶

Warning

Because Elasticsearch automatically tries to rebuild resilience with remaining nodes, the shutdown procedure timing is essential (there are configured safe guards to avoid data displacement by the cluster manager if the cluster is shutting down or starting quickly enough. This is why LMC shutdown should always be done through ansible, and not on each node.

punchplatform-deployer.sh --ssh all 'supervisorctl stop all' -kK

(If your remote system admin account is not the same as your local login, add [-u \<remoteLogin>])

Shut down all PunchPlatform applicative servers¶

punchplatform-deployer.sh --ssh all 'shutdown -h now' -kK