Skip to content

Level 2: Storm

Before going through this guide, ensure you went through the level 1 : getting started deployer guide to get a zookeeper cluster.

This tutorial explains how to add a Storm cluster to an existing platform. It is interesting as Storm requires a zookeeper cluster. We will thus deploy a Storm cluster leveraging the zookeeper cluster you installed in the level 1 guide.

Again, the deployer can be your macos or linux laptop. The starting situation is unchanged.


Configure Storm

Edit first the with your favorite text editor file and include the following content

"storm" : {
    "clusters" : {
        "main": {
            "master" : {
                "servers" : ["server1", "server2", "server3"],
                "cluster_production_address" : "server1"
            "ui" : {
                "servers" : ["server1", "server2", "server3"],
                "cluster_admin_url": "server1:8080"
            "slaves" : ["server1", "server2", "server3"],
            "zk_cluster" : "common",
            "zk_root" : "storm-1.1.3-main",
            "storm_workers_by_punchplatform_supervisor" : 15,
            "workers_childopts" : "-Xmx1G",
            "supervisor_memory_mb" : 8192,
            "supervisor_cpu" : 4
    "install_dir": "/data/opt/apache-storm-2.1.0"

To get more details on each subsection, please read the documentation.

Next edit the punchplatform-deployment.settings file and include the following:

  "storm_version" : "apache-storm-2.1.0",
  "storm_nimbus_nodes_production_interface" : "eth1",

To get more details on each subsection, please read the punchplatform-deployment.settings documentation.


Adapt the properties to suit your environment in particular the zookeeper_nodes_production_interface, as well as unix user and groups.

For example if you use a vagrant box the user is likely to be 'ubuntu' and the network interface 'enp0s8'. In doubt you can find out what is the right network intefaces using the ifconfig command.

Make sure you have the requirements installed on the deployer and target servers. In particular the deployer must have ssh access to the target server.

Check and Generate Your Configurations

First execute this command:

$ --generate-inventory
Or in short:
$ -gi
This command generates a complete set of so-called inventories from your two configuration files. It should output something like this:
$ --generate-inventory
LOG: checking your dependencies...
LOG: all the required packages are installed !

LOG: audit the punchplatform configuration...
LOG: found /Users/dimi/Punch/deployer/conf/
LOG: found punchplatform-deployment.settings: /Users/dimi/Punch/deployer/conf/punchplatform-deployment.settings

LOG: generating deployment configuration from + punchplatform-deployment.settings...
LOG: Following groups and tags can bu used as additional parameters like: --deploy [-l group1,group2] [-t tag1,tag2]
  - group: punchplatform_cluster     tag: N/A
  - group: zookeeper_servers      tag: zookeeper
  - group: storm_servers      tag: storm

LOG: configuration is ready for the deployment !

If that succeeds you can proceed.

Deploy Your Component(s)

From the deployer, make sure you can access your target server using ssh. Depending on your ssh configuration this may require a ssh password. Simply execute the following command. deploy -Kk --tags storm


You will be prompted (or not) for a ssh password. This depends on your ssh configuration.

Check your deployment

ssh server1
sudo systemctl status storm-supervisor
● storm-supervisor.service - Punchplatform Storm-supervisor
   Loaded: loaded (/lib/systemd/system/storm-supervisor.service; enabled; vendor preset: enabled)
   Active: active (running) since Sat 2020-02-15 17:30:48 UTC; 5h 21min ago

As you can see you now have a ready to use Storm cluster. Move on to the next level 3 guide to add an operator environment so that you can define and run punchplatform channels.