Per Stream Destinations
The lumberjack, syslog and syslog udp output nodes all support a configuration option that allows you to have a single node in charge of dispatching the data received on respectively storm stream1 stream2 .. streamN to a (respectively) destination1 destination2 destinationN.
Here is the way to configure this. Let us start with a regular mono-stream example.
{
"type" : "syslog_udp_bolt",
"settings" : {
"destination" : [ {"host" : "127.0.0.1", "port" : 9999}]
},
...
"subscribe" : [
{ "component" : "spout1", "stream" : "logs", "grouping": "localOrShuffle" }
]
}
Say now you have two streams , and you need to send data from to 127.0.0.1:9999 and from logs2 to 127.0.0.1:8888. Here is how to configure it:
{
"type" : "syslog_udp_bolt",
"settings" : {
"destinations" : [
{
"stream" : "logs1",
"destination" : [ {"host" : "127.0.0.1", "port" : 9999}]
},
{
"stream" : "logs2",
"destination" : [ {"host" : "127.0.0.1", "port" : 8888}]
}
]
},
...
"subscribe" : [
{ "component" : "spout1", "stream" : "logs1", "grouping": "localOrShuffle" }
{ "component" : "spout2", "stream" : "logs2", "grouping": "localOrShuffle" }
]
}