Punchlets¶

First we will explore Punchlets.

A Punchlet is a small function in charge of transforming your data. A typical example is log parsing. If you are familiar with Logstash, think of a Punchlet as the filter part of a logstash configuration.

Basics¶

The Standalone ships in with simple examples. Go to the sample folder:

cd $PUNCHPLATFORM_CONF_DIR/samples/punchlets

Then run one as follows:

punchplatform-puncher.sh basic/hello.punch

You will get:

{
  "logs": {
    "age": 22,
    "log": "a sample log",
    "user": "bob"
  }
}

Check the hello.punch file :

// @test(encoding=json) {"logs":{"log":"a sample log"}}
{
    print("Initial log :");
    print(root);
    [logs][user] = "bob";
    print("Adding user :");
    print(root);
    [logs][age] = 22;
    print("Adding age :");
}

Punchlets can be packaged in various ways. Here is an example punchlet that is contained in a YAML file. That makes it easier to document and to define the test input data.

Execute this second example:

punchplatform-puncher.sh advanced/operators_ipmatch.yaml

You will get

{
  "check": true,
  "logs": {
    "log": "172.16.0.2"
  }
}
{
  "check": false,
  "logs": {
    "log": "5.36.18.2"
  }
}

Here is the complete yaml file :

description: >
  Here is a simple example of the ipmatch operator.
  You want to know if an IP is in a list of ranges.
tests:
- logs:
    log: 172.16.0.2
- logs:
    log: 5.36.18.2
resources:
  ranges:
  - 10.0.0.0/8
  - 172.16.0.0/12
  - 192.168.0.0/16
  - 127.0.0.1/32
punchlet: >
  {
    // Retrieve our IP domains from a punch resource.
    // Resources comes from outside the punchlet. In this
    // example it is defined above but in production
    // it typically come from a S3 store, a remote filesystem
    // etc.. 
    Tuple ranges = getResourceTuple("ranges");

    // You can use that resource tuple like any other.
    [check] = ipmatch(ranges).contains([logs][log]);
  }

In there, the code of that particular punchlet is quite simple. It checks if an IP address belongs to some defined range.

{
  Tuple ipRange = getResourceTuple("ranges");
  [check] = ipmatch(ipRange).contains([logs][log]);
}

The resources tuple ipRange fetches the ranges data provided outside your punchlet. In this all-in-one example, that data is actually defined inside the same YAML file:

resources:
  ranges:
  - 10.0.0.0/8
  - 172.16.0.0/12
  - 192.168.0.0/16
  - 127.0.0.1/32

On production platform, such data can come from various files such as CSV, YAML or JSON files.

The Punch language is powerful and comes with a comprehensive online documentation. You will later on see how to invoke it from various stream or batch pipelines.