Skip to content

Overview

Abstract

The punch provides its users with very few concepts to understand and a minimal set of commands to learn. This chapter provides a quick overview of everything you need to understand. Dedicated chapters provides more detailed information on each topic.

Every data processing pipeline, every administrative task is defined as part of a tenant. In each tenant, you further organize your processing units into channels. A channel is a set of task, each defined as a punchline.

Channels may also contain administrative tasks to provide you with your data lifecycle configuration, or third party components such as logstash, or even yours. Whatever mixture of punchlines or third party components you group in a channel, you act on the channel using once simple straight commands : start, stop, reload.

A complete punch platform can easily be understood and presented using a configuration folder layout. Assume for now that configurations are stored on a local filesystem. Here is what it would look like:

└── conf
    ├── resources
    │   ├── groks
    │   ├── mappings
    │   ├── models
    │   └── punchlets
    └── tenants
        ├── customer1
        │   ├── channel
        │   │   ├── admin
        │   │   │   ├── channel_structure.hjson
        │   │   │   └── housekeeping_punchline.hjson
        │   │   └── apache_httpd
        │   │       ├── archiving_punchline.hjson
        │   │       ├── channel_structure.hjson
        │   │       └── parsing_punchline.hjson
        │   └── etc
        └── platform
            ├── channel
            │   └── admin
            │       ├── channel_structure.hjson
            │       └── monitoring_punchline.hjson
            └── etc
  • confThe root folder.
  • resources contain various shared resources your will need. Think of parsers, enrichment files, machine learning models, kibana generic dashboards.
  • tenants contains your per-tenant configuration. Remember everything is defined in a tenant.
    • the reserved platform tenant is used for platform level applications. Some monitoring or housekeeping tasks are typically defined at that level. End user do not have access to this tenant, only the platform administrators.
    • here the customer1 tenant is a fictive example.
  • channels all application are grouped in ways you decide in channels.
  • channel_structure.hjson each channel (for instance 'admin' or 'apache_httpd') is defined using this file. It basically defines the channel content.
  • punchlines.hjson individual tasks are defined by punchlines.

In reality you can have more applications than sketched out here, including third-party apps. But this minimal description suffices for you to understand the punch.

Tip

on simple platform this directory is the top-level root folder holding all the platform files. On production platforms the configuration is stored on some backend and exposed through the REST punch gateway API.