Skip to content

Execution Monitoring

You can monitor the execution of your PML by indicating reporters who will receive PML execution progress information.

PML Configuration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
{
  runtime_id: my-job-id
  tenant: job_tenant
  job:
  [
    // Your PML job here
    {
          type: elastic_batch_input
          component: input
          settings: { }
    }
    {
          type: elastic_batch_input
          component: input
          settings: { }
    }
  ]
  metrics:
  {
    reporters:
    [
      {
        type: elasticsearch
        http_hosts:
        [
          {
            host: localhost
            port: 9200
          }
        ]
      }
      {
        type: console
      }
    ]
  }
}

1
punchplatform-analytics.sh --job myjob.pml -v

Reporters

Elasticsearch

PML progress and execution information will be available in an elasticsearch index (<tenant>-metrics-YYYY-MM-DD)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
{
    type: elasticsearch
    http_hosts:
    [
      {
        host: localhost
        port: 9200
      }
    ]
}

Console

PML progress and execution information will be directly displayed in the standard output.

1
2
3
{
  type: console
}

Kafka

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
"metrics" : {
    "reporters" : [
        {
            "type" : "kafka",
            "bootstrap.servers" : "host1:9092,host2:9092",
            "topic": "<topic_name>"
            "reporting_interval" : 30
        }
    ]
}

Socket

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
"metrics" : {
    "reporters" : [
        "socket_metric_reporter" : {
            "type" : "tcp",
            "reporting_interval" : 30,
            "destination" : [
                {
                    "compression" : true,
                    "host" : "[your_dest_address]",
                    "port" : 9999
                }
            ]
        }
    ]
}

Lumberjack

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
"metrics" : {
    "reporters" : [
        {
            "type" : "lumberjack",
            "metric_document_field_name" : "log",
            "metric_type_field_name" : "metric_type",
            "reporting_interval" : 30,
            "destination" : [
                {
                    "compression" : true,
                    "host" : "target.ip.address",
                    "port" : 9999
                }
            ]
        }
    ]
}

Logger

1
2
3
4
5
6
7
8
9
"metrics" : {
    "reporters" : [
        {
            "type" : "logger",
            "reporting_interval" : 5,
            "format" : "kv"
        }
    ]
}

Command argument

1
$ punchplatform-analytics.sh --tenant mytenant --job myjob.pml -v