org.thales.punch.libraries.storm.bolt.file.buffer

Interface IArchiveBuffer

All Known Implementing Classes:

AbstractArchiveBuffer, AvroParquetArchiveBuffer, CephGzipArchiveBuffer, GzipArchiveBuffer, PlainArchiveBuffer, S3ArchiveBuffer, StreamingGzipArchiveBuffer

public interface IArchiveBuffer

This interface defines the contract for marshaling incoming data (i.e. tuples) into a buffer.

Our first naive implementation accumulates all the tuples in memory up to the final step for constructing a possibly compressed byte array.

New variants perform continuous buffering in a way much more efficient.

Author:

dimi

Method Summary

Modifier and Type	Method and Description
void	close(List<String> successAddresses) Close this archive buffer.
String	getDigest() Return the digest of written bytes
long	getSize() Return the size, compressed if compression enabled, of written bytes
long	getUncompressedSize() Return the size, uncompressed if compression enabled, of written bytes
IArchiveBuffer	prepare(org.thales.punch.libraries.objectstorage.indexing.BatchMetadata metadata) Before using the archive buffer call this.
void	put(Map<String,Object> record) Add a single item in the archive buffer.

Method Detail

put

void put(Map<String,Object> record)

Add a single item in the archive buffer. Watch out here an item means a csv line or one column.

Parameters:

record - the input record

prepare

IArchiveBuffer prepare(org.thales.punch.libraries.objectstorage.indexing.BatchMetadata metadata)

Before using the archive buffer call this. It allows some implementation to complete the preparation, open files etc.

Parameters:

metadata - the provided context information.

Returns:

this instance

getSize

long getSize()

Return the size, compressed if compression enabled, of written bytes

Returns:

compressed size

getUncompressedSize

long getUncompressedSize()

Return the size, uncompressed if compression enabled, of written bytes

Returns:

uncompressed size

close

void close(List<String> successAddresses)

Close this archive buffer. You cannot use it anymore after calling this. You of course MUST call this once done writing to an archive.

Parameters:

successAddresses - in case of success the returned list of where the writes were successful.

getDigest

String getDigest()

Return the digest of written bytes

Returns:

SHA256 Digest