org.thales.punch.libraries.storm.bolt.file.buffer

Class StreamingGzipArchiveBuffer

java.lang.Object

org.thales.punch.libraries.storm.bolt.file.buffer.AbstractArchiveBuffer

org.thales.punch.libraries.storm.bolt.file.buffer.StreamingGzipArchiveBuffer

All Implemented Interfaces:

IArchiveBuffer

public class StreamingGzipArchiveBuffer
extends AbstractArchiveBuffer

A streaming gzip output file archiver. This is the recommended archiver. Writes

Author:

dimi

Field Summary

Fields inherited from class org.thales.punch.libraries.storm.bolt.file.buffer.AbstractArchiveBuffer

destinations, filename, maxWrites, minWrites, pool

Constructor Summary

Constructors

Constructor and Description
StreamingGzipArchiveBuffer(int minWrites, int maxWrites, List<org.thales.punch.libraries.objectstorage.tuples.ArchiveDevice> destinations, org.thales.punch.libraries.record.client.encoder.api.IRecordEncoder<Map<String,Object>> encoder, org.thales.punch.libraries.record.client.encoder.api.IRecordEncoder.RecordFormat format, boolean createRoot)

Method Summary

Modifier and Type	Method and Description
void	close(List<String> successAddresses) Close this archive buffer.
void	exit(Exception cause) Exit archiving.
String	getDigest() Return the digest of written bytes
long	getSize() Return the size, compressed if compression enabled, of written bytes
long	getUncompressedSize() Return the size, uncompressed if compression enabled, of written bytes
IArchiveBuffer	prepare(org.thales.punch.libraries.objectstorage.indexing.BatchMetadata metaData) Before using the archive buffer call this.
void	put(Map<String,Object> record) Add a single item in the archive buffer.

Methods inherited from class org.thales.punch.libraries.storm.bolt.file.buffer.AbstractArchiveBuffer

getBatchOutputPath, handleDeviceFailure

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StreamingGzipArchiveBuffer

public StreamingGzipArchiveBuffer(int minWrites,
                                  int maxWrites,
                                  List<org.thales.punch.libraries.objectstorage.tuples.ArchiveDevice> destinations,
                                  org.thales.punch.libraries.record.client.encoder.api.IRecordEncoder<Map<String,Object>> encoder,
                                  org.thales.punch.libraries.record.client.encoder.api.IRecordEncoder.RecordFormat format,
                                  boolean createRoot)

Method Detail

prepare

public IArchiveBuffer prepare(org.thales.punch.libraries.objectstorage.indexing.BatchMetadata metaData)

Description copied from interface: IArchiveBuffer

Before using the archive buffer call this. It allows some implementation to complete the preparation, open files etc.

Specified by:

prepare in interface IArchiveBuffer

Overrides:

prepare in class AbstractArchiveBuffer

Parameters:

metaData - the provided context information.

Returns:

this instance

getSize

public long getSize()

Description copied from interface: IArchiveBuffer

Return the size, compressed if compression enabled, of written bytes

Returns:

compressed size

getUncompressedSize

public long getUncompressedSize()

Description copied from interface: IArchiveBuffer

Return the size, uncompressed if compression enabled, of written bytes

Returns:

uncompressed size

put

public void put(Map<String,Object> record)

Description copied from interface: IArchiveBuffer

Add a single item in the archive buffer. Watch out here an item means a csv line or one column.

Parameters:

record - the input record

close

public void close(List<String> successAddresses)

Description copied from interface: IArchiveBuffer

Close this archive buffer. You cannot use it anymore after calling this. You of course MUST call this once done writing to an archive.

Parameters:

successAddresses - in case of success the returned list of where the writes were successful.

exit

public void exit(Exception cause)

Description copied from class: AbstractArchiveBuffer

Exit archiving. Basic mode is just log and exit.

Overrides:

exit in class AbstractArchiveBuffer

Parameters:

cause - : Exception raised.

getDigest

public String getDigest()

Description copied from interface: IArchiveBuffer

Return the digest of written bytes

Returns:

SHA256 Digest