Skip to content

PConsole

In this simple tutorial, we will use the PConsole to start and stop a simple kubernetes application.

First download and install the PConsole package as explained here.

Why 'console' ? Because it does not ship with all the components (Zookeeper, Kafka, Elasticsearch...) in contrast to the punch standalone. It only provides:

  • the resources, scripts, runtime binaries to start punch applications.
  • the required java and python APIs to develop custom punchline nodes.

Refer to the channectl, punchlinectl and friends for a description of the available command line punch tools.

PConsole Features

Applications

The PConsole lets you execute the punch applications (punchlinesand plans) nin two modes.

  • In foreground mode using the planctl and punchlinectl commands, the (spark flink or punch) applications will simply run as plain single-process apps. That is useful for developping or testing.
  • In cluster mode: if instead you use the channelctl command, your applications will be submitted to the remote kubernetes cluster.

Important

The PConsole is designed to work with a target runtime: minikube, a real kube, a remote (legacy) punch through the Shiva orchestrator. That is: the punch standalone is a PConsole plus the kafka, clickhouse, elasticsearch and minio database locally installed.

Use the standalone if you want to developp and test your pipelines locally with these components.

Tutorial Example

In this tutorial we will work with a kubernetes punc. Here is an example of the platform.json file that must be configured before installing the PConsole. It describes a platform with the following components already installed :

To operate a kubernetes cluster, the platform.json file must provide the credential and access to the required exposed service. Here is a typical example:

{
  "kubernetes": {
    "config": "$HOME/.kube/config"
  },
  "resources_manager": {
    "elasticsearch": {
      "host": "elasticsearch.doc-store:9200"
    },
    "minio": {
      "host": "http://s3.object-store:9000",
      "access_key": "minio",
      "secret_key": "K@st2020*"
    }
  }
}

Provisioning

If you work with a real kubernetes cluster deployed using kast, make sure to correctly setup and provision your PConsole. In this simple Tutorial we assume you use the default namespace.