Skip to content

Storm

Apache Storm in Punch comes along with official storm binaries and with our extended java binaries.

Description

Check Storm documentation for more information.

Storm binaries

Storm binaries in punch can be break into 3 parts.

  • internal punch binaries
  • external binaries
  • official storm binaries

An overview below:

$ls $PUNCHPLATFORM_INSTALL_DIR
apache-storm-2.3.0  lib  logstash-7.10.2  plugins  resources  spark-2.4.3-bin-hadoop2.7

$tree $PUNCHPLATFORM_INSTALL_DIR/lib
├── punch-archive-app-6.1.0-jar-with-dependencies.jar
├── ...
├── storm
    ├── punch-topology-app-6.1.0-jar-with-dependencies.jar
├── pyspark
│   ├── ...
└── spark
    ├── ...

Notice that extlib is missing in $PUNCHPLATFORM_INSTALL_DIR. This is because no external dependencies were deployed !

Internal Punch binaries

Internal punch binaries are located for storm under:

$PUNCHPLATFORM_INSTALL_DIR/lib/storm

When a patch is done; binaries located under the mentionned directory are substituted by the freshly deployed ones.

External binaries

External binaries are located for storm under:

$PUNCHPLATFORM_INSTALL_DIR/extlib/storm

All external jars deployed by the operator should be under this directory.

These dependencies can later be added to a Punchline with Storm runtime through the use of additional_jars setting.

Patch internal punch binaries for Storm

Refer to this documentation

Add an external library for Storm

Refer to this documentation