org.thales.punch.libraries.storm.bolt.impl

Class AtLeastOnceFileOutput

java.lang.Object

org.thales.punch.libraries.storm.api.BaseProcessingNode

org.thales.punch.libraries.storm.bolt.AbstractFileOutput

org.thales.punch.libraries.storm.bolt.impl.AtLeastOnceFileOutput

All Implemented Interfaces:

Serializable, org.apache.storm.task.IBolt, org.apache.storm.topology.IComponent, org.apache.storm.topology.IRichBolt, org.thales.punch.libraries.punchlang.api.IRuntimeContext

public class AtLeastOnceFileOutput
extends AbstractFileOutput

The "At least once" version of the FileBolt.
In case of a topology failure/restart, duplicated Tuples will be found. Also, when a batch has been written, it will *NEVER* be overwritten so you do not have any issue with filesystem that locks files.
Input must be from Kafka spout. The batch size is configured at the filebolt level. Flushes when reaching batch size or expiration time.

See Also:

AbstractFileOutput, Serialized Form

Field Summary

Fields inherited from class org.thales.punch.libraries.storm.bolt.AbstractFileOutput

archiveFormatSettings, fileNameSupplier, tupleRtt, userStream

Fields inherited from class org.thales.punch.libraries.storm.api.BaseProcessingNode

ackRate, collector, componentId, every, failRate, metricContext, nodeSettings, topologyContext, traversalTime

Constructor Summary

Constructors

Constructor and Description
AtLeastOnceFileOutput(org.thales.punch.libraries.storm.api.NodeSettings boltConfig, List<org.thales.punch.libraries.objectstorage.tuples.ArchiveDevice> archiveUrls, IFileNameSupplier fileNameSupplier, org.thales.punch.libraries.objectstorage.tuples.ArchiveFormatSettings archiveFormatSettings) Ctor

Method Summary

Modifier and Type	Method and Description
Map<String,Object>	getComponentConfiguration()
void	prepare(Map stormConf, org.apache.storm.task.TopologyContext context, org.apache.storm.task.OutputCollector outputCollector) Prepare this bolt.
void	process(org.apache.storm.tuple.Tuple tuple)

Methods inherited from class org.thales.punch.libraries.storm.bolt.AbstractFileOutput

addRuntimeData, createArchiver, flush, getBloomFilterFields, getRuntimeData, getTimestampFromTuple, initMetadata

Methods inherited from class org.thales.punch.libraries.storm.api.BaseProcessingNode

ack, cleanup, declareOutputFields, enrichAndForwardMonitoringTuple, execute, fail, getNewPoint

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AtLeastOnceFileOutput

public AtLeastOnceFileOutput(org.thales.punch.libraries.storm.api.NodeSettings boltConfig,
                             List<org.thales.punch.libraries.objectstorage.tuples.ArchiveDevice> archiveUrls,
                             IFileNameSupplier fileNameSupplier,
                             org.thales.punch.libraries.objectstorage.tuples.ArchiveFormatSettings archiveFormatSettings)

Ctor

Parameters:

boltConfig - Bolt settings

archiveUrls - Url to archive destinations

fileNameSupplier - Supplier for filename (Object storage or Simple filename)

archiveFormatSettings - Specific Batch settings

Method Detail

prepare

public void prepare(Map stormConf,
                    org.apache.storm.task.TopologyContext context,
                    org.apache.storm.task.OutputCollector outputCollector)

Description copied from class: AbstractFileOutput

Prepare this bolt. Use this directly only for testing, and only after setting what you need in terms of configuration.

Specified by:

prepare in interface org.apache.storm.task.IBolt

Overrides:

prepare in class AbstractFileOutput

process

public void process(org.apache.storm.tuple.Tuple tuple)

Specified by:

process in class org.thales.punch.libraries.storm.api.BaseProcessingNode

getComponentConfiguration

public Map<String,Object> getComponentConfiguration()

Specified by:

getComponentConfiguration in interface org.apache.storm.topology.IComponent

Overrides:

getComponentConfiguration in class org.thales.punch.libraries.storm.api.BaseProcessingNode