org.thales.punch.libraries.storm.bolt

Class PunchDispatcherNode

java.lang.Object

org.thales.punch.libraries.storm.api.BaseProcessingNode

org.thales.punch.libraries.storm.bolt.PunchDispatcherNode

All Implemented Interfaces:

Serializable, org.apache.storm.task.IBolt, org.apache.storm.topology.IComponent, org.apache.storm.topology.IRichBolt

public class PunchDispatcherNode
extends org.thales.punch.libraries.storm.api.BaseProcessingNode

The PunchDispatcherNode simply dispatches incoming tuples to different punchlet(s), based on one of the received fields.

This is handy to run small parser topologies assuming that an upfront punchlet or component already enriched the data with the type of log.

Why not deploy several bolts in a single topology instead, and use streams to do the required wiring ? There are two potential reasons that may be important to you.

First the order of message processing will not be guaranteed anymore. Simply because each PunchNode will handle its message in a dedicated thread.

Next, this requires (as just highlighted) more threads. If you are on a very constraint server, you may want to limit yoru topology to the smallest number of threads.

Here is an example configuration :

 {
      "type": "punch_bolt_dispatcher",
      "bolt_settings": {
        "dispatcher_field" : "type",
        "dispatcher_map" : {
          "sourcefire" : {
            "punchlet" : [
              "standard/common/input.punch",
              "standard/common/parsing_syslog_header.punch",
              "standard/sourcefire/parsing.punch",
              "standard/common/geoip.punch"
            ]
          },
          "apache_httpd" : {
            "punchlet_json_resources" : [
              "standard/apache_httpd/http_codes.json"
              ,"standard/apache_httpd/taxonomy.json"
            ],
            "punchlet" : [
              "standard/common/input.punch",
              "standard/common/parsing_syslog_header.punch",
              "standard/apache_httpd/parsing.punch",
              "standard/apache_httpd/enrichment.punch",
              "standard/apache_httpd/normalization.punch",
              "standard/common/geoip.punch"
            ]
          }
        }
      },
 

Author:

dimi

See Also:

Serialized Form

Field Summary

Fields inherited from class org.thales.punch.libraries.storm.api.BaseProcessingNode

ackRate, collector, componentId, every, failRate, metricContext, nodeSettings, topologyContext, traversalTime

Constructor Summary

Constructors

Constructor and Description
PunchDispatcherNode(org.thales.punch.libraries.storm.api.NodeSettings config, Map<String,PunchNode> dispatcher) Ctor

Method Summary

Modifier and Type	Method and Description
void	prepare(Map stormConf, org.apache.storm.task.TopologyContext context, org.apache.storm.task.OutputCollector collector)
void	process(org.apache.storm.tuple.Tuple input)

Methods inherited from class org.thales.punch.libraries.storm.api.BaseProcessingNode

ack, cleanup, declareOutputFields, enrichAndForwardMonitoringTuple, execute, fail, getComponentConfiguration, getNewPoint

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PunchDispatcherNode

public PunchDispatcherNode(org.thales.punch.libraries.storm.api.NodeSettings config,
                           Map<String,PunchNode> dispatcher)

Ctor

Parameters:

config - the bolt configuration

dispatcher - a map of string to punchbolts.

Method Detail

process

public void process(org.apache.storm.tuple.Tuple input)

Specified by:

process in class org.thales.punch.libraries.storm.api.BaseProcessingNode

prepare

public void prepare(Map stormConf,
                    org.apache.storm.task.TopologyContext context,
                    org.apache.storm.task.OutputCollector collector)

Specified by:

prepare in interface org.apache.storm.task.IBolt

Overrides:

prepare in class org.thales.punch.libraries.storm.api.BaseProcessingNode