Skip to content

Gateway

Abstract

The Punchplatform Gateway is a Restful service placed in front of the Punchplatform services such as Elasticsearch, Punchctl client for channels or PML jobs. It provides a transparency access to the Punchplatform features by an endpoint design for any external client.

image

Start

On a fresh standalone, run :

# run in background
punchplatform-gateway.sh --start
# run in foreground
punchplatform-gateway.sh --start-foreground

Logs

Check the Gateway's status with :

punchplatform-gateway.sh --status

In standalone, check the application logs on your file system :

tail -f $PUNCHPLATFORM_CONF_DIR/../external/punchplatform-gateway-6.0.0/logs/punchplatform-gateway.log

Or check the Rest API logs in Kibana in index platform-logs.

Feature redirections

All the endpoints for routing are described in REST API doc

Elasticsearch

Every Elasticsearch cluster is accessible through /es/{cluster_id}.

The redirection respects a request transparency to the Elasticsearch clusters.
Each path matching the pattern /es/{cluster_id}/** will be directly rerouted to the cluster.

Example :

curl GET localhost:4242/v1/mytenant/es/es_search/_cat/indices
curl -XPUT localhost:4242/v1/mytenant/es/es_search/newindex

Channels

Channel management is accessible through /channels.

GET method is used to request channels status, while each POST method is used to execute start and stop actions.

Example :

curl -v GET localhost:4242/v1/mytenant/channels
curl -v -XPOST localhost:4242/v1/mytenant/channels/admin/start
curl -v GET localhost:4242/v1/mytenant/channels/admin
curl -v -XPOST localhost:4242/v1/mytenant/channels/admin/stop

PML

PML job features are accessible through /pml.

It allows a client to save a PML to Elasticsearch, query the saved ones, execute them and request the execution results.

Examples :

# save PML
curl -XPOST localhost:4242/v1/mytenant/pml/save \
  -F file=/@tmp/dataset_generator.pml
# scan saved PMLs
curl GET localhost:4242/v1/mytenant/pml/scan
# execute saved PML
curl -XPOST localhost:4242/v1/mytenant/pml/{pml_id}
# directly execute PML
curl -XPOST localhost:4242/v1/mytenant/pml \
  -F file=@/tmp/dataset_generator.pml
# get PML execution
curl GET localhost:4242/v1/mytenant/pml/{pml_id}/executions/{execution_id}
# delete PML
curl -XDELETE localhost:4242/v1/mytenant/pml/{pml_id}

Puncher

The Puncher tool for punchlets processing is accessible through /puncher.

It allows a client to directly execute a grok or a dissect operator on inputs, or execute a complete punchlet over a log file.

Examples :

# grok operator on input
curl -XPOST localhost:4242/v1/puncher/grok \
  -F input=@/tmp/inputfile \
  -F pattern=@/tmp/patternfile
# dissect operator on input
curl -XPOST localhost:4242/v1/puncher/dissect \
  -F input=@/tmp/inputfile \
  -F pattern=@/tmp/patternfile
# execute punchlet
curl -XPOST localhost:4242/v1/puncher/dissect \
  -F input=@/tmp/inputfile \
  -F logFile=@/tmp/logfile

Security

Authentication forwarding

Gateway will forward any authorization header to Elasticsearch cluster.

The concerned endpoints are :

  • Elasticsearch
  • PML (for storage and execution to Elasticsearch's indices)

All token types supported by Elasticsearch Rest API are also supported by the Punchplatform Gateway.

Abstract

How to get the token?
In the case of standalone with Opendistro, the token is a base64 encoding of the "login:password" chain.
You can generate a token using, for example, the website base64encode.org.
The token for the standalone corresponding to the credentials admin:admin is YWRtaW46YWRtaW4=.

Example :

curl -v GET localhost:4242/v1/mytenant/es/_cat/indices -H "Authorization: Basic YWRtaW46YWRtaW4=" 

yellow open platform-logs-2020.01.28             JVGEA2xsRUWDhNCVn18vdg 5 1   10 0  55.2kb  55.2kb
yellow open .kibana_92668751_admin               MdP8UNobT8SmW3U276K6iQ 1 1    1 0   3.7kb   3.7kb
green  open .kibana_1                            Zq5w1fBtSPeIQvZ45vhdyQ 1 0    0 0    261b    261b
yellow open security-auditlog-2020.01.28         W12oeFkYT7qXc3B_pcREog 5 1   11 0 174.8kb 174.8kb
yellow open platform-metricbeat-6.8.6-2020.01.28 UnTZL_U5QZqMU8bZtao94g 1 1 1479 0 962.9kb 962.9kb
green  open .opendistro_security                 Su-xHUevSL2IarcTfhu-lA 1 0    5 0  25.6kb  25.6kb

SSL

There are two ways to activate SSL for the Punchplatform Gateway :

  1. Client to Gateway
  2. Gateway to endpoints

These features are both independant and disabled in standalone by default, but you can trigger them inside the Gateway configuration file.

SSL for clients to Gateway

A keystore is provided by the standalone in $PUNCHPLATFORM_CONF_DIR/../external/punchplatform-gateway-6.0.0/res/ssl/gateway.keystore

To activate SSL from any clients to Gateway's Rest API, set server.ssl.enabled to true :

vi $PUNCHPLATFORM_CONF_DIR/../external/punchplatform-gateway-6.0.0/conf/application-gateway.yml
# conf for standalone
server:
  address: 127.0.0.1
  port: 4242
  ssl:
    enabled: true
    key-alias: "gateway"
    key-store: "/path/to/gateway.keystore"
    key-store-type: "jks"
    key-store-password: "gateway"
    key-password: "gateway"

You can also create your own keystore with :

keytool -genkey -alias myalias -keyalg RSA -keystore gateway.keystore \
          -validity 3650 -storetype JKS \
          -dname "CN=localhost, OU=Spring, O=Pivotal, L=Kailua-Kona, ST=HI, C=US"
          -keypass changeit -storepass changeit
          -deststoretype pkcs12

Then change the configuration according to your new keystore.

SSL for Gateway to endpoints

Actually, only Elasticsearch endpoints are supported by the gateway to support SSL connection.
Each cluster can be referenced as a protected endpoint with ssl_enbled: true :

elasticsearch:     
  enabled: true   
  data_cluster:                                                    
    cluster_id: "es_search"   
    hosts:                                                         
      - "localhost:9200"  
    settings:
      - "es.index.read.missing.as.empty: yes"
      - "es.nodes.discovery: true"                                                                                                    
    ssl_enabled: true