org.thales.punch.kafka.api

Interface IPartition

All Known Implementing Classes:

AbstractPartition, OffsetTreePartitionImpl, StandardPartitionImpl

public interface IPartition

This interface lets you receive, acknowledge your records, in a way to let you achieve at-least/exactly once semantics.

Internally it keep tracks of acked and failed offsets in tree structures, so as to be able to commit the latest offset in a contiguous sequence. It also lets you implement your own logic.

An example will make it easy to understand. Say you receive records from a partition with the following offset.

 0 1 2 3 4 5 6 7 8 9 ...

You have to ack or fail each one to signal to the IPartition how to proceed. Say you achnowledge

0 1 2 3 5 7 8 9

, but you fail 6 and 4. In general the order in which you ack or fail your record is not ordered. Immediately after failing 6, the kafka position will be rewinded to 6 to reread it again. You will re-receive 6. But not 7 8 9 that you already acked.

Next you fail 4. Again the consumer will restart reading from 4 but will skip sending you records 5 to 9 that are already acked or in the pipe.

That strategy is perfect for simple stream processors that requires at-least once semantics : every record will eventually be processed. Order is unimportant. For some use case this way of working is actually a burden, it would be simpler to brutally rewind everything and restart sending all records from some point in past. If you need something like that you can use the rewind(long) method.

Author:

dimi

Method Summary

Modifier and Type	Method and Description
default void	ack(long offset) Optional operation.
void	attach(Object o) Attach an object to this partition.
void	close() Close this partition
void	commit(long offset, String metaData) Commit an explicit offset for the given partition.
default void	commit(String metaData) Commit the latest acknowledged offset for the given partition.
default void	fail(long offset) Optional operation, fail an offset.
Object	getAttachment()
int	getPartitionNumber()
String	getTopic()
default void	rewind(long offset) Optional operation, rewind the offset to a past position.

Method Detail

getPartitionNumber

int getPartitionNumber()

Returns:

the associated partition number

ack

default void ack(long offset)

Optional operation. Acknowledge an offset

Parameters:

offset - the offset

fail

default void fail(long offset)

Optional operation, fail an offset.

Parameters:

offset - the failed offset

rewind

default void rewind(long offset)

Optional operation, rewind the offset to a past position. The next read will start from there.

Parameters:

offset - the offset value

commit

default void commit(String metaData)

Commit the latest acknowledged offset for the given partition.

Parameters:

metaData - an optional metadata, can be null. If not it will be saved along with the commit

commit

void commit(long offset,
            String metaData)

Commit an explicit offset for the given partition.

Parameters:

offset - the offset to commit. Remember you should always commit +1 the latest processed message (this is the kafka way).

metaData - an optional metadata, can be null. If not it will be saved along with the commit. It should relate to the NEXT data to read and process

attach

void attach(Object o)

Attach an object to this partition. You can then retrieve it using getAttachment().

Parameters:

o - the object to attach

getAttachment

Object getAttachment()

Returns:

the attached object if any

getTopic

String getTopic()

Returns:

the owner topic

close

void close()

Close this partition