User Guide¶
The PunchPlatform is a programmable platform. You code your processing functions using punchlets, deployed as Storm components. To make it easier and less error prone, you also code the generation of configuration file so as to assemble your punchlets in more complex topologies. This guide explains these concepts using simple examples. Mode detailed documentation can be found in the Punch and Storm dedicated documentation:
In this document we explain the basic concepts, for you to understand how to assemble punchlets, spouts and bolts all together to process the data.
Punchplatform Configuration¶
Platform Concepts¶
Channel Concepts¶
Tenant Concepts¶
Additional Concepts¶
Spouts Configuration¶
- The Storm components javadoc documentation provides a detailed description of each spout/bolt configuration.
Bolts Configuration¶
- The Storm components javadoc documentation provides a detailed description of each spout/bolt configuration.
Punch Programming¶
- The javadoc based Punch Documentation covers Punch and Punchlet programming in more details
Shiva Guide¶
Archiving Guide¶
Kibana Guide¶
Batch requests Service Guide¶
Machine-Learning¶
- The javadoc based Analytics Documentation covers Punchplatform Analytics package
Aggregations & Complex Events Processing Guide¶
Alerting Configuration¶
Best Practices¶
HowTos¶
Additional Getting Started¶
Use Cases/PunchPlatform Configuration¶
- HOWTO develop punchlets in Sublime Text
- HOWTO emit multiple events from one event in a punchlet
- HOWTO flush a Kafka backlog
- HOWTO extract logs from Elasticsearch and write them in files
- HOWTO extract massive amount of logs from archiving system
- HOWTO replay logs in Kafka from a specific date
- HOWTO configure a topology to write logs in ceph
- HOWTO reinject documents/logs from archive
- HOWTO configure functional tags
- HOWTO configure EPS reporting functionality
- HOWTO check the presence of logs in ceph
- HOWTO check if my platform receives all the logs
- HOWTO filter out logs
- HOWTO start my first elastalert
- HOWTO start pygregator as a shiva service in a channel
- HOWTO test a parser before go to production
- HOWTO start my first Shiva Cron Job
- HOWTO run a small change between two elasticsearch
Transversal Integration¶
Elasticsearch¶
Machine-Learning¶
Manual Pages¶
- Manual Pages
- punch
- punchplatform-admin.sh
- punchplatform-channel.sh
- punchplatform-env.sh
- punchplatform-kafka-topics.sh
- punchplatform-log-injector.sh
- punchplatform-objects-storage.sh
- punchplatform-analytics.sh
- punchplatform-puncher.sh
- punchplatform-setup-kibana.sh
- punchplatform-setup-grafana.sh
- punchplatform-shiva.sh
- punchplatform-synchronization.sh
- punchplatform-topology.sh
- punchplatform-version.sh
- punchplatform-zookeeper-console.sh
- punchplatform-jobs.sh