Per Stream Destinations
The lumberjack, syslog and syslog udp bolts all support a configuration option that allows you to have a single bolt in charge of dispatching the data received on respectively storm stream1 stream2 .. streamN to a (respectively) destination1 destination2 destinationN.
Here is the way to configure this. Let us start with a regular mono-stream example.
1 2 3 4 5 6 7 8 9 10 11 12 | { "type" : "syslog_udp_bolt", "bolt_settings" : { "destination" : [ {"host" : "127.0.0.1", "port" : 9999}] }, "storm_settings" : { ... "subscribe" : [ { "component" : "spout1", "stream" : "logs", "grouping": "localOrShuffle" } ] } } |
Say now you have two streams , and you need to send data from to 127.0.0.1:9999 and from logs2 to 127.0.0.1:8888. Here is how to configure it:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | { "type" : "syslog_udp_bolt", "bolt_settings" : { "destinations" : [ { "stream" : "logs1", "destination" : [ {"host" : "127.0.0.1", "port" : 9999}] }, { "stream" : "logs2", "destination" : [ {"host" : "127.0.0.1", "port" : 8888}] } ] }, "storm_settings" : { ... "subscribe" : [ { "component" : "spout1", "stream" : "logs1", "grouping": "localOrShuffle" } { "component" : "spout2", "stream" : "logs2", "grouping": "localOrShuffle" } ] } } |