Tuples

Abstract

The punchlang Tuple type lets you manipulate json document with a very compact and intuitive API. This chapter covers the essential features and usages of using the Tuple type.

Assignment

You can write something like:

[person][name] = "bob";

Which will create a person object if is does not exists. I.e. producing:

{
  "person": {
    "name": "bob"
  }
}

Tuples allows you to assign but also to retrieve the content of your JSON. For example you can write:

Tuple tmp;
tmp:[name] = "brenda";
[person][name] = tmp:[name];   

A last important point to note. In some case you want to alter a Tuple content from the top. I.e. you want to completely overwrites its content with something else. You need a way to refer to the top value of the Tuple. Here is how :

{  
    // the ":/" notation refers to the top value. 
    root:/ = "new content";
}

{  
    root = "new content";
}

Because the root variable is a reference to a map structure. If you assign it a new value (here a reference to a String) it will not alter the content of the root Tuple passed to your punchlet. You simply altered the local reference passed to you as an argument.

Tuples and Types

The Tuple supports the JSON basic types. For instances

[number] = 1;
[float] = 3.4;
[boolean] = true;
[string] = "hello";

each instruction here is a valid Punch statement. Tuples stores primitive types as java long, String, boolean, or double. To be safe you should stick to these types. The other way around you can write:

long number       = [number].asLong();
String s          = [name].asString();
boolean condition = [condition].asBoolean();
double rate       = [rate].asDouble();

If you want to be sure of types, you can also use these methods:

if ([number].isLong()) {
    long number = [number];
}
if ([tuple].isEmpty()) { ... }
if ([name].isStringLeaf()) { ... }
if ([long].isLongLeaf()) { ... }
if ([rate].isDoubleLeaf()) { ... }
if ([array].isArray()) { ... }
if ([dict].isTuple()) { ... }
if ([boolean].isBooleanLeaf()) { ... }

You also have the usual operator >, <, >=, <= etc ..

Arrays

A Tuple can contain arrays, just like JSON. It also provide sets. Sets are not sorted and do not contain duplicates. Besides, they share many equivalent methods.

{
    Tuple array;
    array.append("hello").append("beautiful").append("world").append("!");
    print("Resulting array:");
    print(array);

    // Same example with a set
    Tuple set;
    set.add("hello").add("beautiful").add("beautiful").add("world").add("!").add("hello");
    print("Resulting set:");
    print(set);

    // print it
    System.out.println("array size is " + array.size());

    // iterating. You can also use the '.asArray()' or '.elements()' method
    for (Tuple t : array.asCollection()) {
        System.out.println("iterating in array: " + t);
    }

    // Checking for containment
    if (is("hello").inside(array)) {
        System.out.println(" is inside the array");
    }
}

Dynamic fields

Using %{}, You can refer to values stored inside a Tuple to construct or access parts of Tuple dynamically. For example , assuming you have in the root Tuple

{ "user" : "world" }

Then

[hello_%{[user]}] = "bob";

will assign . Punch allows you to also refer to local variables, not only Tuple leaf values. Variables are referred to using the ' $ ' character instead of '% '. Here is the last example using a string variable.

Here a punchlet that you can test with both examples.

cat hello.punch
{
    [user] = "world";
    [hello_%{[user]}] = "bob";

    String w = "mundo";
    [ola_${w}] = "juan";
}

punchplatform-puncher.sh -p hello.punch
{
    "hello_world": "bob",
    "user": "world",
    "ola_": "juan"
}

Strings versus JSON

As said earlier, Punch programming is about manipulating structured documents, not JSON directly (which is just a representation of your Tuples).

There is a subtle difference between both: escaping. Say you receive the following CSV formatted log:

hello,"beautiful,world",!

As you can expect, in the Tuple this is inserted as is, but the quotes have to be escaped when outputting JSON representation.

According to your needs, you may want to manipulate your Tuples escaped or not. Here 's how:

{
    // Print the root content as is.
    // Warning: it looks like JSON, but it's not.
    // Useful to be sure about your input.
    dump(root);
    // Print a JSON representation of your Tuple, with escaping.
    print(root);
    // Get it in a String (raw)
    String myLog = root:[logs][log].asString();
    // Get the associated JSON
    String myLogInJSON = root:[logs][log].toString();
    print(myLog);
    print(myLosInJSON);
}

It prints out:

{logs:{log:hello,"beautiful,world",!}}
{"logs":{"log":"hello,,!"}}
hello,"beautiful,world",!
"hello,,!"

What you typically do in a punchlet is to parse the log using GROK, KV or CSV operators. These operators will work on the unescaped String. For example to perform a CSV extraction you would write :

{
    csv().delim(",").on([logs][log]).into([logs][parsed]));
    print(root);
}

The CSV operator will split the value using the ', ' character as delimiter. By default it will skip any delimiter contained inside inner quotes. I.e. if will not separate 'beautiful ' and 'world '.

"logs": {
  "log": "hello,,!"
  "parsed" : {
      "field0" : "hello",
      "field1" : "beautiful,world",
      "field2" : "!"
  }
}