Elasticsearch and Kibana¶

You may not be familiar yet with Elasticsearch and Kibana. It is a good idea to simply visit your local Kibana at http://localhost:5601.

Here we will explore some Punch monitoring dashboards in Kibana.

First, we need to setup Elasticsearch Templates and Kibana Resources.

Elasticsearch Templates¶

The Punch Standalone comes with Elasticsearch templates. Those templates tells Elasticsearch how to index data properly for each index patterns.

Check out the $PUNCHPLATFORM_CONF_DIR/resources/elasticsearch/templates folder:

├ resources
├── elasticsearch
│   └── templates
│       ├── cyber
│       │   └── mapping_events.json
│       ├── other
│       │   ├── ecs-1.0.0-beta2-template.json
│       │   └── mapping_aggregations.json
│       ├── platform
│       │   ├── pp_mapping_applications.json
│       │   ├── pp_mapping_applicative_monitoring.json
│       │   ├── pp_mapping_archive.json
│       │   ├── pp_mapping_gateway.json
│       │   ├── pp_mapping_metadata.json
│       │   ├── pp_mapping_platform_health.json
│       │   ├── pp_mapping_platform_logs.json
│       │   ├── pp_mapping_platform_monitoring.json
│       │   ├── pp_mapping_topology_metrics.json
│       │   ├── pp_monitoring_default_refresh.json
│       │   └── README_pp_monitoring_mappings.md
│       └── standalone
│           ├── settings_global_standalone.json
│           └── settings_kibana.json

Run the following command to import those templates in Elasticsearch :

punchplatform-push-es-templates.sh -d $PUNCHPLATFORM_CONF_DIR/resources/elasticsearch/templates/ \
    -l http://localhost:9200

You can check it was successful by typing :

curl localhost:9200/_template | jq keys

Important

Some templates are mandatory for the Punchplatform. Those templates are under the platform folder.

They make sure monitoring events generated by the Punch are properly indexed. Other templates are examples for Standalone demos, you can create your own mapping when you create a new channel.

For more information, check Elasticsearch official documentation on [mapping)(https://www.elastic.co/guide/en/elasticsearch/reference/7.8/mapping.html) and templates

Kibana Resources¶

The Punch comes with predefined Kibana dashboards, visualizations and index patterns to quickly start exploring your data. These dashboards are located under the $PUNCHPLATFORM_CONF_DIR/resources/kibana folder.

├── kibana
│   └── dashboards
│       ├── archiving_monitoring
│       │   └── archiving_monitoring.ndjson
│       ├── cyber
│       │   ├── aggregation_mytenant_demo
│       │   │   └── aggregation.ndjson
│       │   ├── cybersecurity_mytenant_demo
│       │   │   └── cybersecurity_mytenant_demo.ndjson
│       │   └── elastic_common_schema
│       │       └── elastic_common_schema_demo.ndjson
│       ├── gateway_monitoring
│       │   └── gateway-monitoring.ndjson
│       ├── kafka_monitoring
│       │   └── kafka-monitoring.ndjson
│       ├── metrics_eps_standalone
│       │   ├── README.md
│       │   └── standalone_eps.ndjson
│       ├── platform_monitoring
│       │   ├── channels_applications_states.ndjson
│       │   ├── channels_monitoring.ndjson
│       │   ├── platform_monitoring.ndjson
│       │   ├── shiva_monitoring.ndjson
│       │   └── shiva_tasks_monitoring.ndjson
│       ├── README_monitoring_dashboards.md
│       ├── spark_monitoring
│       │   └── spark_monitoring_dashboard.ndjson
│       ├── system_monitoring
│       │   └── system-monitoring.ndjson
│       ├── tenants_monitoring
│       │   └── tenants_monitoring.ndjson
│       └── zookeeper_monitoring
│           └── zookeeper-monitoring.ndjson

To import these dashboards you can use this command-line:

punchplatform-setup-kibana.sh --import \
    -l http://localhost:5601

Or import them in Kibana UI :

Go to the Kibana UI
On the left-side panel, go to the "Management > Saved Objects > Import"
Drag-n-drop or select the NDJSON dashboard
Go to the "Dashboard" tab and start exploring your dashboards.

Info

Dashboards from the "*_demo" folders are examples for standalone channels. Others are representative of monitoring dashboards used on production platforms.

Visit the punch dashboards documentation.