Skip to content


The next concept to understand are punchlines. A punchline is a data pipeline, configured to fetch or receive data, process it and push it downstream. Why not just run one ?

cd $PUNCHPLATFORM_CONF_DIR/samples/punchlines/files/csv_to_stdout_and_elasticsearch

Have a look at the csv_to_stdout_and_elasticsearch.yaml file. It is a topology file. It reads data from the local file AAPL.csv, then calls a small processing function AAPL.punch (a punchlet) to convert CSV into JSON, prints that JSON to standard output and inserts it into an elasticsearch cluster.

Go for it !

punchlinectl csv_to_stdout_and_elasticsearch.yaml

You will get:

# [...]
  "logs": {
    "log": {
      "High": 172.0,
      "Low": 170.059998,
      "Volume": 171.509995,
      "Adj Close": 37687500,
      "Close": 171.509995,
      "Date": "2018-01-26",
      "Open": 172.0
# Ctrl+C to quit

The data displayed on your terminal comes from the punchlet that is part of your punchline. Notice the print function on the code below. The punchlet in action simply transforms a CSV input string into a JSON document which in turn will be indexed to elasticsearch.

    if ([logs][log].contains("Date")) {
    if (!csv("Date","Open","High","Low","Close","Volume","Adj Close")
            .into([logs][log])) {
        raise("unexpected format");

At this point, you should have access to your data within Kibana. We will take this opportunity to get familiar with what Kibana has to offer, i.e visualising our data.

  1. Open the kibana GUI in your browser http://localhost:5601
  2. Navigate to the Dev Tools tab on the left side.
  3. Execute the first request shown there: GET /_cat/indices?v. You should see one line with a stock-YYYY.MM.DD index, the one that holds our stock price data.
  4. Now, navigate to the "Stack Management" tab. Click on "index patterns" to create a new index pattern. Use the pattern stock-* and click on "next step". Here, select "@timestamp" as time filter field name.
  5. Navigate to the "Discover" tab. On the Discover menu, at the top, select the stock-* index using the drop-down menu and chose a large time scope.
  6. Now, you should see your data, congratulation !


Mastering Elasticsearch and Kibana is ultimately required to take the best out of your punchplatform. If you are a newcomer: you can consider that an index in elasticsearch is equivalent to a table in a traditional SQL database. Each document you see represent a line in that table, and each field a column. What Kibana does is to allow you to explore and visualise your data easily."

As you see, punchlines are quite simple to understand. They are (very) powerful. You can do all sort of stream computing with them. Now that you have a good understanding of punchlines, let's move on.