Skip to content

Stormline helm

Abstract

The punchline application provides with the light runtime engine necessary to run streaming punchlines. Such punchline is deployed on Kast as a stateless (single process) pod.

Warning

This section describes the punch helm stormline usage only for debugging. His administration (start/status/stop) is wrapped in punch commands so you do not need to use it directly.

Deploy stormline

Warning

Requirements :

  • An access to a Kast cluster

Command example :

helm install my-punchline $PUNCHPLATFORM_HELM_CHARTS/stormline \
    --set-file punchline=/path/to/punchline.json \
    --set-file logger=/path/to/log4j-custom.xml \
    -f settings.yaml \
    --namespace kast

Configuration

Files

File Purpose
punchline The dag topology with the punch components (Required)
logger Custom log4j settings file (Optional)

Punchline File

The punchline file describing the punchline graph of nodes.

Logger File

The logger file sets the logging properties.

Settings

Example :

global:
  imagePullPolicy: "Always"
  imagePullSecret: "mysecret"
  initContainerImage: "gitlab.thalesdigital.io:5005/punch/product/pp-punch/resourcectl:7.0.1-SNAPSHOT"
  containerImage: "gitlab.thalesdigital.io:5005/punch/product/pp-punch/stormline:7.0.1"

  nameOverride: punchline

  service:
    type: NodePort
    containerPort: 9995
    sysloginput:
      port: 9997
      protocol: TCP
    metrics:
      port: 9996
      protocol: TCP

  kafka:
    cluster_name: kafka
    service_port: 9092
    topic: syslog-to-kafka

resource_manager:
  host: "149.202.173.40/gateway"
Field Purpose
resource_manager.host Resource manager URL to download punchline required resources
global.containerImage Stormline docker image reference
global.initContainerImage Resourcectl docker image reference
global.imagePullPolicy Stormline and resourcectl docker image pull policy.
global.imagePullSecret Secret required to pull images from the docker repository

Test punchline

Tests are :

  • Executed in a pod
  • Executed per input and output
  • Weight-ordered

Info

Tests MUST be launched alongside a running release !

Configure test outputs

Example :

# APP
replicaCount: 1

# TESTS
global:
  tests:
    - component: kafka_output

# ID (must be the same as command line one)
configMap:
  - id: my-punchline
    dest: /opt/punch/conf/punchline.json
  - id: my-resolver
    dest: /opt/punch/resolv.yml

Configure test inputs with injection

You may update your punchline with a log injector :

# APP
replicaCount: 1

# TESTS
global:
  tests:
    # syslog input test involves a TCP log injector
    - component: syslog_input
      order: 1
    - component: kafka_output
      order: 2

# ID (must be the same as command line one)
configMap:
  - id: my-punchline
    dest: /opt/punch/conf/punchline.json
  - id: my-resolver
    dest: /opt/punch/resolv.yml
  - id: my-injector
    dest: /opt/punch/conf/injector.json

Warning

In this case, you have to provide and configure a log injector. Refer to :

Launch test

Since your release is running, upgrade it with your tests.

Info

If a Punch Log Injector is involved for an input test, you have to provide it with --set-file my-resolver=resolv.yml

helm upgrade my-punchline $PUNCHPLATFORM_HELM_CHARTS/punchline \
    --set-file my-punchline=punchline.json --set-file my-resolver=resolv.yml \
    --set-file my-injector=injector.json \
    -f platform.yml  -f channel.yml -f punchline.yml \
    --namespace kast

Example :

helm test my-punchline -n kast

NAME: my-punchline                                                                                                                   
LAST DEPLOYED: Fri Mar 12 11:54:26 2021                                                                                               
NAMESPACE: kast                                                                                                                       
STATUS: deployed                                                                                                                      
REVISION: 7                                                                                                                           
TEST SUITE:     my-punchline-kafka-output-test                                                                                       
Last Started:   Fri Mar 12 11:54:38 2021
Last Completed: Fri Mar 12 11:54:55 2021
Phase:          Succeeded
TEST SUITE:     my-punchline-syslog-injector-test
Last Started:   Fri Mar 12 11:54:29 2021
Last Completed: Fri Mar 12 11:54:38 2021
Phase:          Succeeded