Punch Documentation
Punch 8.1.13
Search
K

Welcome

Welcome to the Punch product documentation for the 8.x release.

Overview

Executive Summary
Data processing solutions such as IoT, log management, real-time monitoring, or alerting require configuring and then orchestrating many small applications, some ever running, some periodically executing. Coding, configuring, and managing all these applications quickly becomes complex and costly, even on the best cloud provider platforms.
Punch dramatically simplifies this work by combining a few best-of-breed patterns:
  • low code / serverless design. You only develop business functions, nothing else. The rest is configuration-based and automatically generated.
  • Automatic application generation using powerful templating features. This is possible because Punch provides a wealth of execution runtimes and data connectors to accommodate all types of data processing applications: big (using Spark or Flink) or small (using Rust, web assembly, or Java). In all cases, it is highly efficient.
  • Cloud Native. Punch installs with very few dependencies on top of your Kubernetes platform.
For Architects
Punch is a production-ready Kubernetes native FaaS/Serverless framework initially developed for log management use cases. It provides a rich SDK that includes many advanced parsing and enrichment functions.
Punch has many standard log parsers and lets you write additional business functions in Python, PySpark, Spark, Java, or Rust. A proprietary programming language, Punchlang, is also available. Punchlang dramatically simplifies the development of common transformation functions.
To design and deploy industrial machine learning and analytics applications, an integrated Jupyter engine is also provided. It significantly facilitates the design of Spark and Python machine-learning applications.

Punch At a Glance

Punch At a Glance
Punch is a simple and cost effective solution to design data processing applications. Examples are log management or IoT applications that ships data from remote site(s) or device(s) to one or several central platform. Punch is business agnostic: it provides several data processing engines to execute your business functions. You can deploy them on small devices, on intermediate gateways or as part of large scale central platforms.
Punch provides key backend services: configuration and device management. These services are cloud (Kubernetes) native.

What Is Punch Used For?

Here is a typical example where Punch provides an observability solution. Collecting logs or events from various external sources, parsing, normalising, and enriching that data on the fly, and generating alerts based on several strategies, from executing detection rules to more sophisticated machine learning functions.
Observability Use Case

How Does It Work?

Punch is an extra simple Kubernetes native solution that focuses on providing a minimal set of services to enable you to put in place your business use cases quickly. The four pillars of Punch are:
  • Data Lineage: Data comes from edge devices or remote platforms up to your central database. Each step (transport, processing, archiving, indexing, parsing, etc.) is a...
  • Pipelines: pipelines chain source and sink connectors and execute in between your...
  • Functions: shipped on a function registry. Here, you have parsers, rules, models, and custom functions written in Rust, Java, Python, or Spark.
  • Configuration: All that is ultimately represented by (YAML) configuration files. Punch configuration management provides a versioned, simple, and robust administrative board to manage these files. Most of them are automatically generated.
The four Punch Pillar

What is Special About It?

The strength of Punch is the following
  • Your functions are deployed in pipelines automatically generated from rules or patterns. Punch is not a heavy designer to draw complex, unmanageable ETLs.
  • Punch provides several runtimes for executing functions: Python, Rust, Java, and Spark, and can leverage others (Flink). You only focus on your functions. Or not, because Punch provides lots of them. Punch runtime runs on your edge devices as well.
  • Punch does not ship with tons of dependencies. It comes with no dependency. As long as there is a cloud storage solution on your cloud or on-premise Kubernetes platform, Punch installs itself with a few helm commands.
  • Punch is secured: it provides advanced features to free you from dealing with secrets, credentials, and the like.
  • Vendor locking is limited: you only depend on Punch data function APIs. All your data is in your databases. Punch leverages the ECS elastic common schema normalization, but you are free to choose your own. Punch APIs are simple and will not pollute your business code.
  • Punch is SQL-friendly: all pipelines are expressed using tables and columns. Streaming SQL engines are shipped with it.
  • Last Punch comes with detection rule compilers to automatically compile and run Sigma rules, simple or complex.

Product

The product and commercial offerings are described in the product information space.

Architecture

Punch is designed as a Kubernetes native solution to interoperate with key open-source technologies such as Elasticsearch, OpenSearch/OpenDashboard, S3 stores, ClickHouse, Spark, and Kafka. Punch provides simple and powerful deployment options for your function to act on the data on the fly or at rest.
Once you have your functions, Punch deploys them in various runtime engines. These engines are exposed as Kubernetes native applications. You interact with these using Custom Resource Definition files. Behind the scenes, Punch maps these to Java, Python, PySpark, or WebAssembly runtimes. All you have to do is define a processing graph using source and sink data connectors to read and write data from/to various places and insert your functions in the middle. These graphs are called punchlines.
Punchlines cover various functional use cases: data collection and transport from edge to central platforms, data parsing, filtering and enrichment, real-time alerting, or machine learning. Combined with cutting-edge data visualization capabilities, multi-tenancy, and security, Punch lets you quickly put a Punchline on production in the cloud or on-premise infrastructures.
Cybersecurity, system or application monitoring and predictive analysis are typical Punch sweet spots. It can serve virtually all types of business verticals: retail, healthcare, manufacturing, financial services, and government.

Components

Punch applications are shipped as a set of container images and helm charts. Punch images can be found either on the Punch public GitHub package repository, or on demand. The helm charts are publicly available on a dedicated GitHub repository.
The main Punch components are:
  • Punchline engines to execute your graphs of functions.
  • Punch Artifact Server to package, manage, and distribute your models, functions, and their various resources.
  • Punch Kubernetes Operators to expose a REST API and to manage the lifecycle of Punch applications.
  • Visualization plugins to extract data or provide user feedback.

Software Development Kit

Punch provides several toolkits to develop, test, and package your log parsers and functions.

Starters

Punch starters is a set of sample functions freely available from the punchplatform starters GitHub repository. They should be a starting point when developing new functions for your punchlines.

Kooker

Kooker is a lightweight all-in-one Docker-based K8s+ Punch local deployer. Using Kooker you can run a complete Punch system on your laptop in minutes. Refer to the Kooker public GitHub repository.