Skip to content

Track 3 : Channels

Abstract

This track briefly explains how channels are defined to run applications of different types,

including streaming punchlines, periodic tasks and standard punch housekeeping/monitoring applications

Punchlines/Applications in channels

You now have a good understanding of punchlines. These are nothing else than applications. Applications are grouped in channels to ease their management.

Channels are organised as part of 'channels' folder, that is part of each tenant.
Each channel folder must have a channel_structure.yaml file that defines the channel content.

Read the Channel configuration documentation to know about the channel structure and the Storm/Shiva variants of application running.

Exercise : a channel for your punchline

Pick a streaming punchline that receives logs and that indexes them, from track 1 Then create a channel folder and its channel_structure.yaml file, in order to run it on a chosen shiva machine.

Key information

  • To know the names of the shiva cluster, you can look into the platform inventory file 

     jq '.shiva.clusters | keys' $PUNCHPLATFORM_PROPERTIES_FILE

  • To know where your punchline will be running (and listening for logs), specify as placement tag the hostname of the machine. The hostnames can be found from the inventory file: 

    jq '.shiva.clusters | to_entries[] | (.key,(.value.servers | keys))' $PUNCHPLATFORM_PROPERTIES_FILE

  • because your streaming punchline must run forever, it needs no schedule

A solution
version: '6.0'
start_by_tenant: true
stop_by_tenant: true
applications:
- name: listener
  runtime: shiva
  cluster: common
  reload_action: kill_then_start
  command: punchlinectl
  args:
  - start
  - --punchline
  - listener.yaml
  shiva_runner_tags:
  - common

Starting your channel

You can now operate your 'mypipeline' channel with

channelctl -t mytenant start --channel mypipeline

Start and test your channel

  • To centralize your punchline metrics, change the reporter type to 'elasticsearch' (instead of 'console').
  • To avoid using to much memory, add the following section to your punchline:
  settings:
   topology.worker.childopts: "-server -Xms350m -Xmx350m"
   topology.max.spout.pending : 1000

  • test your punchline locally

  • start your channel

  • check status of your channel

  • using Kibana '[PTF / Channels Monitoring] Shiva tasks events log' dashboard, check that you receive the logs of your punchline

  • using Kibana '[PTF Channels Monitoring] Channels applications health and restarts' dashboard, check that you see the location of your streaming app.

  • inject logs fom the operator machine towards your punchline on the input machine

  • check that you receive your logs in kibana

This said, the punch also provides you many other kinds of applications such as :

  • Platform Monitoring applications.
  • Channels Monitoring
  • logstash, a popular log processor application
  • ElastAlert, an applicattion to run alerting rules
  • Plans that are capable of scheduling periodic applications.
  • Your own applications should you need to extend the punch with external tools or scripts.

Monitoring your channel with a standard application

Exercise: Monitor your channels

  • Copy and simplify the "monitoring" channel of the platform tenant, changing it so as to run the channels monitoring standard application:

    • es_topologies_metrics_index setting need to be changed to retrieve the metrics from your tenant, not the platform tenant
    • reporters.index_name must be changed to write the monitoring result in an index associated to your tenant, not the platform one

  • using Kibana '[PTF Channels Monitoring] Channels applications health and restarts' dashboard, check that you see the health of your app.

And there are many other metrics to monitor your channels.

Part of them are used in standard dashboards (Ack/Fails, processing times...). Other you can use to build your process-monitoring dashboards (Kafka queues processing backlog).

Implicit punchline settings

Your platform administrator may have overriden some of your punchline settings.

To see your punchline actual content, user the resolve command : 

punchlinectl -t mytenant resolve -p mypunchline.hjson

Have a look at the metrics section (in which you have put an "elasticsearch" reporter, with a 60s period).

Where do this elasticsearch cluster id and different period comes from ?

Have a look at the deployed platform resolver configuration:

cat $PUNCHPLATFORM_RESOLV_FILE

Stop guessing what it does. Everything is explained here... but the choices have been made by the platform integrator/deployer.

It allows to reduce the platform-specific settings in all your pipeline files, and can help run files coming from other platforms in your test platform (especially a standalone, that has much less clusters than your production platform.)