public class Punchlet extends PunchletContext
{
[es_type] = \"apache\";
}
That simple punchlet will be executed for each receiving json document, passed as the (so-called) root tuple. What it does is simply to add an 'es_type' property with value "apache" to that root document. Compact punchlets are fine for simple stateless punchlets, only acting on the document content. Say for example you want a punchlet that counts the number of documents it receives, and every 1000 documents, insert a property in the traversed document.
public class MyPunchlet extends Punchlet {
long count = 0L;
public void execute(Tuple root) {
count ++;
if ((count % 1000) == 0) {
[root][counter] = count;
}
}
Check out the examples, you can do many things like sending alerts, compute statistics, generate new documents, and more.
Punchlets can also be injected with resources : plain files, json or properties files. Json files are provided to your punchlets directly as tuples. For example say you have a file called locations.json containing
[
{"location": {"code": "FR", "name": "France"}, "interval": ["10.0.0.0", "10.0.0.30"]},
{"location": {"code": "PT", "name": "Portugal"}}, "interval": ["10.0.0.31", "10.0.0.60"]}
]
If you put that file in the (adequate) resource directory, you can refer to
it in your punchlet directly as a tuple. The next example uses that tuple to
perform a lookup by interval using the findByInterval(Tuple)
operator.
{
Tuple tuple;
findByInterval(locations).on("10.0.0.3").into(tuple);
// tuple contains {"location": {"code": "FR", "name": "France"}, "interval": ["10.0.0.0", "10.0.0.30"]}
}
Note that a punchlet can be injected with additional json resource tuples, i.e. tuple that hold the content of static json files, containing configuration data. These are automatically injected as fields in your punchlet.
can be expressed using a compact syntax
Punch provide operators for performing various complex tuple transformations. Operators are invoked from punchlets using a short notation. For example the key-value operator is invoked like this.
String log = "name=\"bob\" age=23"; Tuple tuple; kv().on(log).into(tuple);
the result of running this gives you the Tuple 'tuple' holding
{"age":23,"name":"bob"}
In this page you will find the invocation method for all Punch operators. Refer to the documentation of each operator for further options.
Modifier and Type | Field and Description |
---|---|
static ThreadLocal<Punchlet> |
_threadLocal
this thread local stores the punchlet itself
|
static org.apache.logging.log4j.Logger |
LOG |
child, code, eventCount, metricRegistry, r, uuidGenerator, world
Modifier | Constructor and Description |
---|---|
protected |
Punchlet(RuntimeContext runtimeContext)
Ctor
|
Modifier and Type | Method and Description |
---|---|
boolean |
__isChained()
some punchlet are made of a chain of several punchlets.
|
void |
activate()
The activate method is called after startup, and before receiving input
tuples.
|
Punchlet |
addEventCallback(PunchletEventCallback callback)
This method is not meant to be executed from within a punchlet.
|
void |
attach(String key,
Object value)
Attach a resource to this punchlet for later reuse.
|
BugOperator |
bug() |
static CefOperator |
cef() |
CidrRangeOperator |
cidrRange() |
CidrRangeOperator |
cidrRange(String url)
Deprecated.
|
Object |
coalesce(Object... args)
Return the first non empty object in a list.
|
ContainsOperator |
contains(String resourceName,
String... keys)
Return a
ContainsOperator operator. |
ContainsOperator |
contains(Tuple map,
String... keys)
Return a
ContainsOperator operator. |
static ConvertorOperator |
convert(String input)
Return a convertor operator, transforming XML or JSON strings into tuples.
|
static ConvertorOperator |
convert(Tuple input)
Return a convertor operator, transforming XML or JSON strings into tuples.
|
static CsvOperator |
csv(String... fields) |
static long |
currentTimeMillis() |
static long |
currentTimeNanos() |
static DateOperator |
date() |
static DateOperator |
date(String dstDateFormat)
Return a date operator with a destination time format.
|
static DateOperator |
date(String dstDateFormat,
String srcDateFormat)
Return a date operator with a destination and source time format.
|
static DissectOperator |
dissect(String pattern)
Return a
DissectOperator for a given pattern. |
static void |
dump(Tuple t)
Dump to stdout a plain String representation of a tuple.
|
FetchCsvOperator |
fetchCsv()
Return a fetchCsv operator.
|
FetchCsvOperator |
fetchCsv(String url)
Return a fetchCsv operator.
|
FetchJsonOperator |
fetchJson()
Return a fetchJson operator.
|
FetchJsonOperator |
fetchJson(String url)
Deprecated.
|
FindByIntervalOperator |
findByInterval(Tuple map)
Return a FindByIntervalOperator, acting on the passed json map.
|
static FindByIntervalOperator |
findByInterval(Tuple map,
String key)
Return a FindByIntervalOperator, acting on the passed json map.
|
static FindByIntervalOperator |
findByInterval(Tuple map,
String min,
String max)
Return a
FindByIntervalOperator operator with min and
max property interval boundaries. |
FindByKeyOperator |
findByKey(String resourceName,
String... keys)
Return a
FindByKeyOperator operator. |
FindByKeyOperator |
findByKey(Tuple map,
String... keys)
Return a
FindByKeyOperator operator. |
GeoOperator |
geo()
Return a ready to use geo operator.
|
static GeoIpOperator |
geoip() |
static GeoOperator |
geoOperator()
Get GeoOperator to make geoOperation
|
Object |
getAttachement(String key) |
String |
getCode()
Return the original punchlet code.
|
static String |
getHostIp() |
static String |
getHostName() |
ResourceOperator |
getResource(String resourceName) |
String |
getResourceString(String name)
Return a resource string value.
|
Tuple |
getResourceTuple(String name)
Return a resource json tuple.
|
String |
getUniqueMessageId()
Return a unique id ready to be used as an ElasticSearch (or any other database) unique key.
|
static GrokOperator |
grok(String pattern)
Return a grok operator for the corresponding grok pattern
|
HttpOperator |
http(String url)
Returns an http operator.
|
IpMatchOperator |
ipmatch(String path)
This variant takes in the name of your resource tuple.
|
IpMatchOperator |
ipmatch(Tuple resource)
the ipmatch method lets you check if an IP address is contained in a list.
|
IpRangeOperator |
ipRange()
Use this method to get an IPV4 or IPV6 range operator.
|
IpRangeOperator |
ipv4Range()
Deprecated.
|
IpRangeOperator |
ipv4Range(String url)
Deprecated.
|
IpRangeOperator |
ipv6Range()
Deprecated.
|
IpRangeOperator |
ipv6Range(String url)
Deprecated.
|
static IsOperator |
is(Object o)
Called directly by the dsl at runtime.
|
boolean |
isAsynchronous()
Some punchlets generate asynchronous events.
|
static KvOperator |
kv() |
org.apache.logging.log4j.Logger |
logger()
You can log message using standard java logging.
|
static MailOperator |
mail()
Get a mail operator to parse mail headers and content.
|
com.codahale.metrics.Meter |
meter(String name)
Return a new com.codahale.metrics.Meter, registered to metrics registry
(if any) using the passed name.
|
static MoveOperator |
move(Tuple tuple)
Get a move operator to move a tuple from one place to another.
|
NifiOperator |
nifi()
Get a new nifi operator.
|
static void |
print(String s)
print a string to standard output.
|
static void |
print(Tuple... tuples)
Pretty print a tuple.
|
static void |
raise(String message)
Raise a punch exception.
|
static void |
remove(Tuple tuple)
Remove a tuple.
|
Punchlet |
setMetricContext(org.thales.punch.libraries.metrics.api.IMetricContext metricsContext)
Associate a metric context with this punchlet.
|
String |
sha1(Object... values)
Return a SHA1 hash on the given arguments.
|
static void |
sleep(long ms)
Sleep for some milliseconds
|
static SplitOperator |
split() |
static SyslogHeaderOperator |
syslogHeader() |
Object[] |
toArray(Object... args)
Return an array of arbitrary object.
|
static boolean |
toBoolean(Tuple tuple)
The toDouble operator returns a long value contained in a tuple.
|
static double |
toDouble(Tuple tuple)
The toDouble operator returns a double value contained in a tuple.
|
static ToFlatTupleOperator |
toFlatTuple() |
static ToKvOperator |
toKv() |
static long |
toLong(Tuple tuple)
The toDouble operator returns a long value contained in a tuple.
|
static void |
toLowerCase(Tuple... tuples)
Change tuple(s) content to lower case
|
static void |
toUpperCase(Tuple... tuples)
Change tuple(s) content to upper case.
|
Long |
tryEpochMs(Tuple t)
Convert a date to an epoch timestamp in seconds.
|
Long |
tryEpochMs(Tuple t,
long defaultValue)
Convert a date to an epoch timestamp in seconds.
|
Long |
tryEpochSec(Tuple t)
Convert a date to an epoch timestamp in millisecond.
|
Long |
tryEpochSec(Tuple t,
long defaultValue)
Convert a date to an epoch timestamp in millisecond.
|
static Tuple |
tuple()
Return a new tuple
|
static Tuple |
tuple(Object... elements)
Construct a new tuple.
|
static UnflatTupleOperator |
unflatTuple()
Unflat a tuple.
|
ValidateSchemaOperator |
validateSchema(String... schemas) |
ecount, every, execute, getPath, getPersistentId, getProperties, getRuntimeId, getWorldTuple, invoke, setCode, setPath, setPersistentId, setProperties, setRuntimeId
public static final org.apache.logging.log4j.Logger LOG
public static final ThreadLocal<Punchlet> _threadLocal
protected Punchlet(RuntimeContext runtimeContext)
runtimeContext
- the associated runtime contextpublic static CefOperator cef()
CefOperator
operator.public static ConvertorOperator convert(String input)
input
- the input stringpublic Long tryEpochSec(Tuple t)
t
- the input tuplepublic Long tryEpochMs(Tuple t)
t
- the input tuplepublic Long tryEpochSec(Tuple t, long defaultValue)
t
- the input tupledefaultValue
- the default valuepublic Long tryEpochMs(Tuple t, long defaultValue)
t
- the input tupledefaultValue
- the default valuepublic static ConvertorOperator convert(Tuple input)
input
- the input tuple, expected to hold a string leafpublic static CsvOperator csv(String... fields)
fields
- the field name to name extracted valuesCsvOperator
operator.public static DateOperator date()
public static DateOperator date(String dstDateFormat)
dstDateFormat
- a standard time formatpublic static DateOperator date(String dstDateFormat, String srcDateFormat)
dstDateFormat
- a standard time formatsrcDateFormat
- a standard time formatpublic static long currentTimeMillis()
public static long currentTimeNanos()
public static DissectOperator dissect(String pattern)
DissectOperator
for a given pattern.pattern
- your dissect patternpublic static void dump(Tuple t)
t
- the tuple to dumppublic static FindByIntervalOperator findByInterval(Tuple map, String key)
The passed tuple must contain an array of elements each containing an interval with min and max (long) value. From there the findByInterval operator builds an efficient data structure to perform lookups for all elements whose interval encloses a given value.
map
- the source json interval tree.key
- the key indicating the interval in thereFindByIntervalOperator
operator.public static FindByIntervalOperator findByInterval(Tuple map, String min, String max)
FindByIntervalOperator
operator with min and
max property interval boundaries.map
- the input mapmin
- low rangemax
- high rangepublic static GeoIpOperator geoip()
GeoIpOperator
public static String getHostIp()
public static String getHostName()
public static GrokOperator grok(String pattern)
pattern
- a valid grok expressionpublic static IsOperator is(Object o)
o
- the object to check for containmentpublic static KvOperator kv()
public static MoveOperator move(Tuple tuple)
tuple
- the tuple to movepublic static MailOperator mail()
public static GeoOperator geoOperator()
public static void print(String s)
s
- the stringpublic static void print(Tuple... tuples)
tuples
- the tuple to printpublic static void raise(String message)
message
- a message associated the resulting exceptionpublic static void remove(Tuple tuple)
tuple
- the tuple to removepublic static void sleep(long ms)
ms
- the delay in millisecondspublic static SplitOperator split()
public static SyslogHeaderOperator syslogHeader()
SyslogHeaderOperator
.public static boolean toBoolean(Tuple tuple)
Tuple.asDouble()
method, a tuple with a string
representation of a double is accepted.tuple
- the input tuple, expected to contain a boolean value, "true"
or "false"PunchRuntimeException
- if the tupel contain other types of valuepublic static double toDouble(Tuple tuple)
Tuple.asDouble()
method, a tuple with a string
representation of a double is accepted.tuple
- the input tuple, expected to contain a double value or a valid
double String representation.PunchRuntimeException
- if the tupel contain other types of valuepublic static ToFlatTupleOperator toFlatTuple()
public static ToKvOperator toKv()
public static long toLong(Tuple tuple)
Tuple.asLong()
method, a tuple with a string
representation of a double is accepted.tuple
- the input tuple, expected to contain a long value or a valid
long String representation.PunchRuntimeException
- if the tupel contain other types of valuepublic static void toLowerCase(Tuple... tuples)
tuples
- one or several tuplespublic static void toUpperCase(Tuple... tuples)
tuples
- one or several tuplespublic static Tuple tuple()
public static Tuple tuple(Object... elements)
elements
- the element(s) to be transformed.
If you have more than one an array tuple will be constructedpublic static UnflatTupleOperator unflatTuple()
public boolean __isChained()
public void activate()
public Punchlet addEventCallback(PunchletEventCallback callback)
Register a callback to this punchlet. This is used for asynchronous style punchlets that generate events asynchronously, for example a punchlet with a time based siddhi rule. These events are represented as Tuples.
If you run punchlet from a java applicatin using the punchlet library, you must implement this in order to re-invoke yor punchlet with the raised events.
callback
- the callback to be notified upon fired eventspublic void attach(String key, Object value)
key
- the resource keyvalue
- the corresponding valuepublic BugOperator bug()
public Object coalesce(Object... args)
A tuple will be returned only if it is not empty
args
- the input array of tuples or objectspublic ContainsOperator contains(String resourceName, String... keys)
ContainsOperator
operator.
This variant takes the name of a json resource. The contains operator will throw a PunchRuntimeException if no tuple is defined for the resource name
resourceName
- the name of the json resourcekeys
- the keys to look for.public ContainsOperator contains(Tuple map, String... keys)
ContainsOperator
operator.
Note that the ContainsOperator operator does not require precompilation. It uses a JIT strategy so that repeated calls with the same key will result in efficient lookup.
map
- input mapkeys
- the key to look forpublic FindByIntervalOperator findByInterval(Tuple map)
The passed tuple must contain an array of elements each containing an interval with min and max (long) value. From there the findByInterval operator builds an efficient data structure to perform lookups for all elements whose interval encloses a given value.
map
- the source json interval tree.FindByIntervalOperator
operator.public FindByKeyOperator findByKey(String resourceName, String... keys)
FindByKeyOperator
operator.
This variant takes the name of a json resource. The findByKey operator will throw a PunchRuntimeException if no tuple is defined for the resource name
resourceName
- the name of the json resourcekeys
- the keys to look for.public FindByKeyOperator findByKey(Tuple map, String... keys)
FindByKeyOperator
operator.
Note that the FindByKeyOperator operator does not require precompilation. It uses a JIT strategy so that repeated calls with the same key will result in efficient lookup.
map
- input mapkeys
- the key to look forpublic Object getAttachement(String key)
key
- the key of a previously attached resourcepublic String getCode()
public Tuple getResourceTuple(String name)
Resource files must be associated to your punchlet with a path relative to the "$PUNCHPLATFORM_CONF_DIR/resources/punch/" directory.
The name of the resource is the resource file name without the file extension. For example, if you have in your topology :
"bolt_settings" : {
"punchlet_json_resources" : [
"some/location/your_resource.json"
],
"punchlet" : "some/location/your_punchlet.punch"
},
Then in your punchlet you should use :
Tuple t = getResourceTuple("your_resource");
name
- the tuple namePunchRuntimeException
- if the json resource is not found in your contextpublic String getResourceString(String name)
Resource files must be associated to your punchlet with a path relative to the "$PUNCHPLATFORM_CONF_DIR/resources/punch/" directory.
The name of the resource is the resource file name with the file extension if any. For example, if you have in your topology :
"bolt_settings" : {
"punchlet_file_resources" : [
"some/location/your_resource.json"
],
"punchlet" : "some/location/your_punchlet.punch"
},
Then in your punchlet you should use :
String s = getResourceString("your_resource.json");
name
- the resource original file namePunchRuntimeException
- if the string resource is not found in your contextpublic FetchCsvOperator fetchCsv(String url)
FetchCsvOperator
.url
- the remote resource file URL.public FetchCsvOperator fetchCsv()
FetchCsvOperator
.@Deprecated public CidrRangeOperator cidrRange(String url)
CidrRangeOperator
documentation.
Prefer the {cidrRange()
variant.
url
- the remote or local resource file url.public CidrRangeOperator cidrRange()
@Deprecated public IpRangeOperator ipv4Range()
IpRangeOperator
documentation.url
- the remote or local resource file url.IpRangeOperator
.@Deprecated public IpRangeOperator ipv6Range()
IpRangeOperator
documentation.url
- the remote or local resource file url.IpRangeOperator
.@Deprecated public IpRangeOperator ipv4Range(String url)
IpRangeOperator
documentation.
This variant requires you provide on your platform the ipv4 csv file.
url
- the remote or local resource file url.IpRangeOperator
.@Deprecated public IpRangeOperator ipv6Range(String url)
IpRangeOperator
documentation.
@Deprecated prefer using the ipRange()
method.
url
- the remote or local resource file url.IpRangeOperator
.public IpRangeOperator ipRange()
@Deprecated public FetchJsonOperator fetchJson(String url)
FetchCsvOperator
documentation.
This method is deprecated prefer the {fetchJson()
variant.
url
- the remote resource file URL.public FetchJsonOperator fetchJson()
FetchCsvOperator
documentation.public HttpOperator http(String url)
url
- - the remote urlpublic String getUniqueMessageId()
The returned identifier is both unique and compact.
public IpMatchOperator ipmatch(String path)
path
- the resource tuple.IpMatchOperator
public IpMatchOperator ipmatch(Tuple resource)
IpMatchOperator
documentation for examples.resource
- the resource tuple.IpMatchOperator
public boolean isAsynchronous()
PunchletEventCallback
to be notified.public org.apache.logging.log4j.Logger logger()
if (logger().isInfoEnabled()) {
logger().info("an info message {}", [message].size());
}
public com.codahale.metrics.Meter meter(String name)
name
- the name of the meterpublic String sha1(Object... values)
values
- the values to concatenate and hash.public NifiOperator nifi()
nifi().addOutputRoutes("success").into(root)
NifiOperator
.public Object[] toArray(Object... args)
[array] = toArray(1,2,3);
args
- the objects to be returned as an arraypublic GeoOperator geo()
public Punchlet setMetricContext(org.thales.punch.libraries.metrics.api.IMetricContext metricsContext)
metricsContext
- the provided metric contextpublic ValidateSchemaOperator validateSchema(String... schemas)
public ResourceOperator getResource(String resourceName)
Copyright © 2022. All rights reserved.