Skip to content

5.3.0 to 5.4.0

This document explains what configuration changes MUST be performed during a PunchPlatform update from version 5.3.0 to 5.4.0

General changes

Important

Before upgrading it is recommended to stop all your topologies. If this is a concern to you, make sure you have each topology file have a name property that matches the file name. See an example below.

Configuration files

Topology Files

The name property that appears in the topology file is now deprecated. Simply remove it from your topologies. If you keep it, you will have warning messages but it will not prevent the topologies to be submitted.

Before performing the upgrade it is advised to ensure that the name property matches the topology file name. Say you have a topology called input_topology.json, make sure it contains:

1
2
3
{
  "name" : "input_topology"
}

If you have some disrepency, the punchctl command may not be able to deal with 5.3.0 running topologies. Either stop all topologies before upgrading, or make sure the names match the file names as just explained.

ElasticSearch Spout and Bolt

The configuration name credential has been fixed to credentials for basic authentication :

1
2
3
4
5
6
{
  "credentials": {
    "token": "bXlVc2VyOm15UGFzc3dvcmQ=",
    "token_type": "Basic"
  }
}

Operator Environment section

In the punchplatform-deployment.settings, use the right zip name. Since the version 5.4.0, the artifacts punchplatform-operator-environment-dataanalytics and punchplatform-operator-environment has been merged together. Take a look into your archives deployer directory, you should only find one zip named punchplatform-operator-environment-5.4.0.zip (no dataanalystics suffix).

If you previously used the dataanalytics version, no worries, everything will continue to work as expected with this renamed version.

Here is the quick example:

1
2
3
4
5
# update this line
"punchplatform_operator_environment_version": "punchplatform-operator-environment-dataanalytics-X.Y.Z"

# with this one
"punchplatform_operator_environment_version": "punchplatform-operator-environment-5.4.0"

Monitoring

PunchPlatform API v2

The Elasticsearch monitoring endpoint /punchplatform/api/v2 has been removed. As a replacement, we set up a new index platform-health-[YYYY-MM-DD] that store the high level platform monitoring state as a time serie. Refer to the dedicated monitoring guide.

Note that all the "details" fields has been removed. Here is an example of the Kafka subsection:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# Before
"kafka": {
  "health_code": 0,
  "health_name": "green",
  "clusters": {
    "local": {
      "brokers": {
        "0": {
          "health_code": 0,
          "health_name": "green",
          "details": {
            "listener_security_protocol_map": {
              "PLAINTEXT": "PLAINTEXT"
            },
            "endpoints": [
              "PLAINTEXT://localhost:9092"
            ],
            "jmx_port": -1,
            "port": 9092,
            "host": "localhost",
            "id": "0",
            "version": 4,
            "timestamp": "1562655762744"
          }
        }
      },
      "health_code": 0,
      "health_name": "green"
    }
  }
}
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
# After
"kafka": {
  "health_code": 0,
  "health_name": "green",
  "clusters": {
    "local": {
      "brokers": {
        "0": {
          "health_code": 0,
          "health_name": "green"
        }
      },
      "health_code": 0,
      "health_name": "green"
    }
  }
}

If you need to get more details about a specific technology, use the platform-monitoring-current index.

Metrics

Two fields of the documents from platform-monitoring-current and platform-monitoring-[YYYY-MM-DD] have changed.

The fields "metricset.name" and "metricset.type" have been merged into a single field named "name" and a new field "type" has been added. Here is an example:

1
2
3
4
5
6
7
8
9
# Before
{
  "@timestamp": "2019-07-09T07:02:42.789Z",
  "metricset": {
    "name": "broker",
    "type": "kafka"
  },
  ...
}
1
2
3
4
5
6
7
# After
{
  "@timestamp": "2019-07-09T07:02:42.789Z",
  "name": "kafka.broker",
  "type": "platform",
  ...
}