org.thales.punch.libraries.storm.bolt.file.buffer

Class AbstractArchiveBuffer

java.lang.Object

org.thales.punch.libraries.storm.bolt.file.buffer.AbstractArchiveBuffer

All Implemented Interfaces:

IArchiveBuffer

Direct Known Subclasses:

AvroParquetArchiveBuffer, PlainArchiveBuffer, StreamingGzipArchiveBuffer

public abstract class AbstractArchiveBuffer
extends Object
implements IArchiveBuffer

Field Summary

Fields

Modifier and Type	Field and Description
protected List<org.thales.punch.libraries.objectstorage.tuples.ArchiveDevice>	destinations
protected String	filename
protected int	maxWrites
protected int	minWrites
protected String	pool

Constructor Summary

Constructors

Constructor and Description
AbstractArchiveBuffer(int minWrites, int maxWrites, List<org.thales.punch.libraries.objectstorage.tuples.ArchiveDevice> destinations)

Method Summary

Modifier and Type	Method and Description
protected void	exit(Exception cause) Exit archiving.
protected String	getBatchOutputPath() Return full batch path from metadata.
protected void	handleDeviceFailure(String stage, String device, Exception e, int remaining) When device fails to open, write or close, the device is considered down.
IArchiveBuffer	prepare(org.thales.punch.libraries.objectstorage.indexing.BatchMetadata metaData) Before using the archive buffer call this.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.thales.punch.libraries.storm.bolt.file.buffer.IArchiveBuffer

close, getDigest, getSize, getUncompressedSize, put

Field Detail

minWrites

protected final int minWrites

maxWrites

protected final int maxWrites

destinations

protected final List<org.thales.punch.libraries.objectstorage.tuples.ArchiveDevice> destinations

filename

protected String filename

pool

protected String pool

Constructor Detail

AbstractArchiveBuffer

public AbstractArchiveBuffer(int minWrites,
                             int maxWrites,
                             List<org.thales.punch.libraries.objectstorage.tuples.ArchiveDevice> destinations)

Method Detail

prepare

public IArchiveBuffer prepare(org.thales.punch.libraries.objectstorage.indexing.BatchMetadata metaData)

Description copied from interface: IArchiveBuffer

Before using the archive buffer call this. It allows some implementation to complete the preparation, open files etc.

Specified by:

prepare in interface IArchiveBuffer

Parameters:

metaData - the provided context information.

Returns:

this instance

handleDeviceFailure

protected void handleDeviceFailure(String stage,
                                   String device,
                                   Exception e,
                                   int remaining)

When device fails to open, write or close, the device is considered down. This method checks that the minimum required devices limit is not reached. If below limit, exit archiving. I above, only log a warning.

Parameters:

stage - open,write or close

device - device failing

e - exception raised

remaining - remaining devices count

getBatchOutputPath

protected String getBatchOutputPath()

Return full batch path from metadata. Add pool name before batch_id.

Returns:

full batch path

exit

protected void exit(Exception cause)

Exit archiving. Basic mode is just log and exit.

Parameters:

cause - : Exception raised.