Skip to content

punchplatform-topology.sh

Synopsis

punchplatform-topology.sh path_to_topology_file

punchplatform-topology.sh -m cluster --start --stop --status --topology topology.json

Description

punchplatform-topology.sh provides the high level command to start, stop or query storm topologies. Remember a channel is made up of one or several topologies. Each is defined by a topology json configuration, under the corresponding tenant/channel configuration directory. punchplatform-topology.sh is invoked for each topology by punchplatform-channel.sh but you are free to use it directly.

A running topology is identified by a topology-id made of the tenant, the channel, the name of the topology, and prefixed with the submission timestamp. For example 'sampleTenant_apache_parser_<timestamp>'

With a single argument, punchplatform-topology.sh laucnhes the topology in foreground, using the Storm local mode. That makes it easy to check or run topologies locally.

Options

  • --topology <topology.json>:

    • operate on the topology defined by the topology.json file.
  • --topology-id <topology-id>:

    • operate on the topology identified by the provided topology identifier
  • --storm-cluster-id <storm-cluster-id>:

    • submit the topology to the storm cluster identified by an id. A punchplatform mau run several storm cluster, each with distinct roles. Check the punchplatform.properties file where each storm cluster is defined and configured. topology-id. You can supply only a prefix, you will then operate over all ;atching topologies.
  • --start:

    • start the topology. If the corresponding topology is already running, it is not started. To can force the start of the same topology with the force option. You should not need this option. It can be used to start a new version of the same topology before stopping the running version to achieve minimal service interruption.
  • --start-foreground:

    • run the topology run in foreground. If you use that option with a cluster topology (see the --mode ) option, only the topology building and submission to Storm will be executed in foreground. Even so this is handy to easily catch up issues at build time.
  • --stop:

    • stop the topology.
  • --status:

    • list all the matching topologies together with their runtime status. The return status is 0 if at least one ACTIVE topology is found.
  • -m, --mode <mode>:

    • Set the execution mode, either 'local' or 'cluster'. The local mode runs the topology embedded into a local process. The topology is then not visible from the storm cluster, and will not appear in the Storm UI. Use this mode to run non scalable topologies that require to be executed on dedicated servers, for example a syslog listening topology. In contrast, the cluster mode will submit the topology to the Storm cluster and will be executed wherever the Storm Nimbus daemon decide. The resulting topology is then managed by storm, and visible through the Storm UI.
  • -v, --verbose:

    • Dumps detailed informations
  • -s, --silent:

    • Disable console output.
  • jvm opts:

    • Additional Java opts can be provided before the command argument, and will be passed to the starting jvm. Note however that these options will not be passed to the remote Storm managed worker jvm. These options make only sense for local topologies or for the local jvm building the topology and sending it to the cluster.
  • -aj [:][:...]

    • add additional jars to the classpath of the topology launcher. This is useful to provide additional bolts, spouts or java classes to be used in advanced punchlets. You have to ensure that all needed dependencies are also provided if they are not provided as part of the standard punchplatform library (punchplatform-command-cli*with-dependencies.jar) WARNING : Because your jars will be loaded in the same classloader space as the punchplatform-provided spouts, bolts and all dependencies classes, you are responsible for ensuring the compatibility of the jars you are adding, with the punchplatform and storm classes and dependencies versions. If you also want your topology to be run in a storm cluster (either through this command, or through punchplatform-channels.sh --start), you also will have to ensure your additional jar(s) are stored in the extlib/ directory of the storm cluster installation directory on ALL nodes of the cluster (supervisor nodes and the operator nodes)

Examples

  • To start the channel 'apache' of tenant 'sampleTenant' in the cluster
1
punchplatform-topology.sh --start --mode cluster --start $PUNCHPLATFORM_CONF_DIR/tenants/sampleTenant/apache/input_topology.json
  • start a topology using local storm engine
1
punchplatform-topology.sh ./topology.json
1
punchplatform-topology.sh --start-foreground -m local -t ./topology.json
  • start a topology using punch light storm engine
1
punchplatform-topology.sh --start-foreground -m light -t ./topology.json

Diagnostics

The punchplatform-topology utility exits 0 on success, and >0 if an error occurs.

Environment

The following environment variables affect the execution of punchplatform-topology.sh:

  • PUNCHPLATFORM_CONF_DIR

    • The PunchPlatform_CONFDIR environment variable indicate the directorywhere tenant and channel configuration files are stored. In particular the logback.xml logging configuration files is the one used when topologies are run in foreground, or using the local mode. Refer to that file to understand the various loggers that you can activate to trace issues or output metrics.
  • punchplatform.properties

    • The punchplatform.properties file contains fields that will be used to generate the detailed configuration of your channel, such as the zookeeper root, the kafka cluster identifiers, etc .... It is located in PUNCHPLATFORM_CONF_DIR.
  • NOCOLOR

    • If defined, disable the use or colored output.

See also

punchplatform-deployer.sh or punchplatform.properties

Bugs

No known bugs.