Platform Monitoring Shiva task¶
Abstract
This chapter explains how to configure a Shiva task running the platform health monitoring service.
This permanent-running task (with internal periodic activation) is needed to allow external monitoring system to retrieve a synthetic Healh document for any punchplatform (LTR or Back-Office)
For more information about PunchPLatform platform health api, please refer to Monitoring Guide
Overview¶
The punch platform monitoring service is in charge of monitoring Kafka, Shiva, Storm, ElasticSearch, Zookeeper, Spark and to produce "platform-components monitoring documents" (finally stored in platform-monitoring- Elasticsearch indices), plus a synthetic "platform Health Document" (finally stored in platform-health- Elasticsearch indices). .
Depending on the platform-level reporters configured in the punchplatform.properties, the documents produced by this task will be either directly written to Elasticsearch (if on a back-office) or written to a kafka topic, to be forwarded later to the back-office and its Elasticsearch back-end (for LTR platforms monitoring).
Configuration¶
The PunchPlatform Platform Monitoring service is a task that
is run in the Shiva task scheduler. This requires first to add the following job to the channel channel_structure.json
descriptor file:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | { "version" : "5.0", "start_by_tenant" : true, "stop_by_tenant" : true, "jobs" : [ { "type" : "shiva", "name" : "monitoring", "command" : "platform-monitoring", "args": [ "platform-monitoring-configuration.json" ], "resources": [ "platform-monitoring-configuration.json" ], "cluster" : "common", "shiva_runner_tags" : ["standalone"] } ] } |
Along with an platform-monitoring-configuration.json
file to define your settings. This files contains the following fields:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | { "monitoring_interval": 10, "services": [ "kafka", "shiva", "storm", "elasticsearch", "zookeeper", "spark" ], "credentials": { "es_search": { "user": "admin", "password": "admin" }, "another_cluster": { ... } }, "reporters": [ { "cluster_name": "es_search", "type": "elasticsearch", "index_monitoring": "test-monitoring", "index_health": "test-health" } ] } |
This sample configuration means that Kafka, Shiva, Storm, ElasticSearch, Zookeeper, Spark will be monitored every 10 seconds.
Tip
Remember that if you update your configuration, you must restart your channel and save your configuration.
Parameters¶
Mandatory¶
-
monitoring_interval
(integer)The interval in seconds between two health check of services.
-
services
(string array)The list of service we want to monitor.
values:"kafka"
,"shiva"
,"storm"
,"elasticsearch"
,"zookeeper"
or"spark"
Optional¶
-
credentials
(map of credentials)This is a map of credentials for one or many ElasticSearch clusters. You will need this extra configuration Opendistro Security is enabled on your platform.
-
reporters
(list of reporters)This is a list of reporters. If this field is empty or missing, the platform monitoring uses reporters configured in punchplatform.properties
Try It¶
On a standalone deployment, the Platform monitoring service is
included in the configuration of the admin channel. To
activate the Platform Monitoring service, you must ensure that the
admin
channel is started.
1 2 | punchctl --tenant mytenant configuration --push punchctl --tenant mytenant start --channel admin |