org.thales.punch.libraries.storm.spout.impl

Class FileSpoutWatcher

java.lang.Object

org.apache.storm.topology.base.BaseComponent

org.apache.storm.topology.base.BaseRichSpout

org.thales.punch.libraries.storm.api.BaseInputNode

org.thales.punch.libraries.storm.spout.AbstractFileInput

org.thales.punch.libraries.storm.spout.impl.FileSpoutWatcher

All Implemented Interfaces:

Serializable, org.apache.storm.spout.ISpout, org.apache.storm.topology.IComponent, org.apache.storm.topology.IRichSpout

public class FileSpoutWatcher
extends AbstractFileInput
implements org.thales.punch.libraries.storm.api.ISpout

The "Watcher" implementation of the File Spout.

This Spout look for a specified path directory and read any created file in near-realtime.

To use this Spout, set the "watcher" option to "true" in the Spout configuration.

For example, the following configuration will act as a shell command "tail -f" on every file in the '/path/to/directory' folder with a '.log' extension.

 
 {
   "type": "file_spout",
   "spout_settings": {
     "codec": "line",
     "watcher": true,
     "watcher.process_existing_files": true,
     "read_file_from_start": false,
     "tail": false,
     "path": "/path/to/directory",
     "path.regex": ".\*\.log"
   },
   "storm_settings": {
     "component": "filespout",
     "publish": [
       ...
     ]
   }
 }
 
 

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.thales.punch.libraries.storm.spout.AbstractFileInput

AbstractFileInput.Item

Field Summary

Fields inherited from class org.thales.punch.libraries.storm.spout.AbstractFileInput

failed, inputQueue, inputQueues, orderedRead, path, pathRegexPattern, processAlreadyExisting, queueSize

Fields inherited from class org.thales.punch.libraries.storm.api.BaseInputNode

collector, exitCondition, latencyRecordSender, loadController, metricContext, myself, nodeSettings

Constructor Summary

Constructors

Constructor and Description
FileSpoutWatcher(org.thales.punch.libraries.storm.api.NodeSettings config) Ctor.

Method Summary

Modifier and Type	Method and Description
void	open(Map stormSettings, org.apache.storm.task.TopologyContext topologyContext, org.apache.storm.spout.SpoutOutputCollector collector)

Methods inherited from class org.thales.punch.libraries.storm.spout.AbstractFileInput

ack, drainNextLogs, emitNextLine, fail, getListFiles, nextTuple, readOneFile

Methods inherited from class org.thales.punch.libraries.storm.api.BaseInputNode

close, deactivate, declareOutputFields, getPublishedStreams, regulate, sendLatencyRecord

Methods inherited from class org.apache.storm.topology.base.BaseRichSpout

activate

Methods inherited from class org.apache.storm.topology.base.BaseComponent

getComponentConfiguration

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.thales.punch.libraries.storm.api.ISpout

registerNextTupleCallback

Methods inherited from interface org.apache.storm.spout.ISpout

ack, activate, close, deactivate, fail, nextTuple

Methods inherited from interface org.apache.storm.topology.IComponent

declareOutputFields, getComponentConfiguration

Constructor Detail

FileSpoutWatcher

public FileSpoutWatcher(org.thales.punch.libraries.storm.api.NodeSettings config)

Ctor.

Parameters:

config - the spout settings

Method Detail

open

public void open(Map stormSettings,
                 org.apache.storm.task.TopologyContext topologyContext,
                 org.apache.storm.spout.SpoutOutputCollector collector)

Specified by:

open in interface org.apache.storm.spout.ISpout

Overrides:

open in class AbstractFileInput