punchplatform-puncher.sh
Synopsis¶
punchplatform-puncher.sh [options]
Description¶
Develop, test and run punchlets and parsers.
Command¶
-
-c,--code <arg>
- Use an inline punchlet code
-
-g,--grok <arg>
- Change default grok-pattern resources directory (resource/punch/pattern)
-
-h,--help
- Print help and usage
-
-i,--input-fields <arg>
- Use another input fields key (e.g. [logs][others])
-
-I,--input-file <arg>
- Read log samples from a file
-
-it,--interactive
- Interactive mode, the punchlet will wait for stdin inputs
-
-j,--json-input
- Give a json formatted string as root input to the punchlet
-
-o,--output-fields <arg>
- Use another output fields key (e.g. [a][b])
-
-O,--output-file <arg>
- Write your output to this file
-
-p,--punchlet <arg>
- Punchlet file (.punch)
-
-q,--quiet
- Quiet mode, root tuple will not be printed out
-
-r,--resource <arg>
- Punchlet resource file, .json file or directory
-
-t,--run-unit-test <arg>
- Run a unit test
-
-tk,--tick <arg>
- Processing tick frequency in milliseconds
-
-u,--create-unit-test <arg>
- Generate unit test files
Testing Punchlets¶
Using the puncher you can define unit test for your punchlets. A unit test is a json file that contains three sections. One defines the punchlet(s) you want to test. If you put several ones, the chain of punchlet will be considered.
The input
section lets you define an input log. The output
section
defines what the output parsed log shoud look like.
Here is a simple example:
{
"punchlets" : [
"parser.punch"
],
"input": {
"logs": { "data" : "color=red city=Rome uri=https://punchplatform.com" }
},
"output": {
"includes": {
"logs": {
"data" : "color=red city=Rome uri=https://punchplatform.com",
"log" : {
"color": "red",
"city": "rome",
"proto": "https",
"host": "punchplatform.com",
"uri": "https://punchplatform.com",
"color_code": 1
}
}
}
}
}
To define and package such unit tests, refer to the (parser development guide)[../Contribution_Guide/Parser_Developper/Testing_parsers.md].
Examples¶
Interactive mode, the puncher will wait for user input and output the result for each new line:
punchplatform-puncher.sh -it -c '{[a][b] = "test";}'
Compile multiple punchlets and use [logs][data]
as input field:
punchplatform-puncher.sh -i [logs][data] -p p1.punch -p p2.punch
Use a json as punchlet(s) input. It allows more complex inputs based on multiple fields:
punchplatform-puncher.sh -j '{"logs":{"data":"My log"}}' -p p1.punch
Quiet mode, do not output the final root
tuple:
punchplatform-puncher.sh -q -c '{[test] = "test"}'
The Puncher can read input from stdin. Feel free to pipe it your sample inputs:
echo "my log" | punchplatform-puncher.sh -c '{;}'
punchplatform-puncher.sh -c '{;}' <<< "my log"
cat log_sample.txt | punchplatform-puncher.sh -p p1.punch
punchplatform-puncher.sh -p p1.punch < log_sample.txt
Set input and output files:
punchplatform-puncher.sh -I log_sample.txt -O dir/output.txt -p p1.txt
Limit the speed processing using a frequency tick. Here, input logs will be processed every 2 seconds (value in milliseconds):
punchplatform-puncher.sh -I log_sample.txt -p p1.punch -tk 2000
Test grok or dissect pattern
punchplatform-puncher.sh -gi "AAA000DDDD" -gp "(?<queue_id>[0-9A-F]{10,11})"
punchplatform-puncher.sh -di 'hello world' -dp '%{a} %{b}'