Skip to content

API Reference

GET /v1/platform/properties

Get punchplatform.properties file.

Response

Status

Status Code Reason
200 Successful operation

Example request

curl --location --request GET 'http://localhost:4242/v1/platform/properties'

GET /v1/platform/tenant

Get tenant used by the gateway.

Response

Body

Field Description
tenant Tenant set in Gateway settings

Status

Status Code Reason
200 Successful operation

Example request

curl --location --request GET 'http://localhost:4242/v1/platform/tenant'

Example response

{
    "tenant": "mytenant"
}

POST /v1/puncher/{operator}

Execute a punchlet which using the given operator

Request

Path variable

Field Description
operator Operator to use ("grok", "dissect" or "punchlet")

Form

Field Description
input File containing log samples
pattern File contaning the pattern

Response

Status

Status Code Reason
200 successful operation
500 Invalid input or pattern

Example request

curl --location --request POST 'http://localhost:4242/v1/puncher/dissect' \
--form 'input=@/my_dissect_folder/input' \
--form 'pattern=@/my_dissect_folder/pattern'

DELETE /v1/{tenant}/archives

Delete all existing archives for a tenant

Request

Path variables

Field Description
tenant Tenant name

Response

Status

Status Code Reason
200 successful operation

Example request

curl --location --request DELETE 'http://localhost:4242/v1/mytenant/archives'

GET /v1/{tenant}/archives/{id}

Download an archive generated by a punchline

Request

Path variables

Field Description
tenant Tenant name
id Archive id to download

Response

HTTP Status

Status Code Reason
200 Successful download
404 Archive not found

Example request

curl --location --request GET 'http://localhost:4242/v1/mytenant/archives/my_archive'

DELETE /v1/{tenant}/archives/{id}

Delete an archive

Request

Path variables

Field Description
tenant Tenant name
id Archive id to delete

Response

Status

Status Code Reason
200 Successful download
404 Archive not found

Example request

curl --location --request DELETE 'http://localhost:4242/v1/mytenant/archives/my_archive'

GET /v1/{tenant}/channels

Get all tenant channels status

Request

Path variables

Field Description
tenant Tenant name

Response

Body

Field Description
children Array of valid channels
children[].name Name of the channel
children[].status Status of the channel
errors Array of invalid channels
errors[].name Name of the channel
errors[].status Status of the channel

Status

Status Code Reason
200 successful operation

Example request

curl --location --request GET 'http://localhost:4242/v1/mytenant/channels'

Example response

{
    "children": [
        {
            "name": "logstash",
            "status": "STOPPED"
        },
        {
            "name": "admin",
            "status": "STOPPED"
        },
        {
            "name": "apache_httpd",
            "status": "STOPPED"
        }
    ],
    "errors": []
}

GET /v1/{tenant}/channels/{channel}

Get status of a specific channel

Request

Path variables

Field Description
tenant Tenant name
channel Channel name

Response

Body

Field Description
status Channel status
jobs List of jobs of the channel
jobs[].cluster Job cluster name
jobs[].name Job name
jobs[].runtimeId Job related path
jobs[].status Job status

Status

Status Code Reason
200 successful operation
404 Channel not found

Example request

curl --location --request GET 'http://localhost:4242/v1/mytenant/channels/admin'

Example response

{
    "status": "ACTIVE",
    "jobs": [
        {
            "cluster": "common",
            "name": "admin/common/elasticsearch-housekeeping",
            "runtimeId": "punchplatform/mytenant/channels/admin/elasticsearch-housekeeping",
            "status": "ACTIVE"
        },
        {
            "cluster": "common",
            "name": "admin/common/archives-housekeeping",
            "runtimeId": "punchplatform/mytenant/channels/admin/archives-housekeeping",
            "status": "ACTIVE"
        },
        {
            "cluster": "common",
            "name": "admin/common/channels-monitoring",
            "runtimeId": "punchplatform/mytenant/channels/admin/channels-monitoring",
            "status": "ACTIVE"        
        }
    ]
}

POST /v1/{tenant}/channels/{channel}/{action}

Request

Path variables

Field Description
tenant Tenant name
channel Channel name
action Action to operate on the channel ("start" or "stop")

Response

Status

Status Code Reason
201 Successful action
404 Channel doesn't exist

Example request

curl --location --request POST 'http://localhost:4242/v1/mytenant/channels/admin/start'

GET /v1/{tenant}/conf

List all tenant related files and directories in configuration folder

Request

Path variables

Field Description
tenant Tenant name

Response

Body

Field Description
PATH.parent Parent folder name
PATH.path Relative path
PATH.size Folder/File size
PATH.isFile true if PATH is a file, else false
PATH.canRead Able to read file/folder content
PATH.children Map of subfile / subfolder content

Status

Status Code Reason
200 successful operation

Example request

curl --location --request GET 'http://localhost:4242/v1/mytenant/conf'

Example response

{
    "tenants": {
        "parent": "/conf",
        "path": "/tenants",
        "size": 4096,
        "isFile": false,
        "canRead": true,
        "children": {
            "mytenant": {
                "parent": "/conf/tenants",
                "path": "/tenants/mytenant",
                "size": 4096,
                "isFile": false,
                "canRead": true,
                "children": {
                    "test_file": {
                        "parent": "/conf/tenants/mytenant",
                        "path": "/tenants/mytenant/test_file",
                        "size": 162,
                        "isFile": true,
                        "canRead": true,
                        "children": {}
                    }
                }
            }
        }
    }
}

GET /v1/{tenant}/conf/{path}

Get a file from configuration folder

Request

Path variables

Field Description
tenant Tenant name
path Path to file

Response

Status

Status Code Reason
200 successful operation
400 "path" given is a directory
404 File not found

Example request

curl --location --request GET \
'http://localhost:4242/v1/mytenant/conf/tenants/mytenant/test_file'

POST /v1/{tenant}/extraction

Create extraction

Request

Path variables

Field Description
tenant Tenant name

Body

Field Description
description Extraction description
format Extraction format ("csv", "json")
index Elasticsearch index
fields List of fields to extract
fields[].name Field name
fields[].type Field type
filters Elastic filters to apply
tenant Tenant name
### Response
#### Status
Status Code Reason
200 successful operation

Example request

curl --location --request POST 'http://localhost:4242/v1/mytenant/extraction' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
--header 'Content-Type: application/json' \
--data-raw \
'{
     "description": "log-extraction",
     "format": "csv",
     "index": "platform-logs-*",
     "fields": [
         {
             "name": "@timestamp",
             "type": "date"
         },
         {
             "name": "content.message",
             "type": "string"
         }
     ],
     "filters": {},
     "tenant": "mytenant"
 }'

GET /v1/{tenant}/punchline

List punchlines saved.

Request

Path variables

Field Description
tenant Tenant name

Response

Body

Field Description
[].id Punchline ID
[].punchline Punchline file content
[].version Punchline version
[].platformId Punchline platform ID
[].@timestamp Punchline save timestamp

Status

Status Code Reason
200 successful operation

Example request

curl --location --request GET 'http://localhost:4242/v1/mytenant/punchline'

Example response

[
    {
        "id": "BtDHgnEByFYZVYsBXB30",
        "punchline": {
            "runtime_id": "BtDHgnEByFYZVYsBXB30",
            "metrics": {
                "reporters": [
                    {
                        "http_hosts": [
                            {
                                "port": 9200,
                                "host": "localhost"
                            }
                        ],
                        "type": "elasticsearch",
                        "index_name": "mytenant-metrics"
                    }
                ]
            },
            "job": [
                {
                    "settings": {
                        "input_data": [
                            {
                                "name": "phil",
                                "musician": false,
                                "age": 21,
                                "friends": [
                                    "alice"
                                ]
                            },
                            {
                                "name": "alice",
                                "musician": true,
                                "age": 23,
                                "friends": [
                                    "dimi"
                                ]
                            },
                            {
                                "name": "dimi",
                                "musician": true,
                                "age": 53,
                                "friends": [
                                    "phil",
                                    "alice"
                                ]
                            }
                        ]
                    },
                    "component": "input",
                    "publish": [
                        {
                            "stream": "data"
                        }
                    ],
                    "description": "The batch_input node simply generates some data.\nYou simply write your data inline, it convert it as Dataset<Row>",
                    "type": "dataset_generator"
                },
                {
                    "settings": {
                        "truncate": false
                    },
                    "component": "show",
                    "subscribe": [
                        {
                            "component": "input",
                            "stream": "data"
                        }
                    ],
                    "type": "show"
                }
            ],
            "type": "spark",
            "tenant": "mytenant"
        },
        "version": -1,
        "platformId": "my-unique-platform-id",
        "@timestamp": "2020-04-12T11:37:15.766Z"
    }
]

GET /v1/{tenant}/punchline/executions

Get all tenant related punchlines executions.

Request

Path variables

Field Description
tenant Tenant name

Response

Body

Field Description
id Execution ID
timestamp Last event timestamp
duration Execution duration
tenant Tenant name
event Execution last event

Status

Status Code Reason
200 successful operation

Example request

curl --location --request GET \
'http://localhost:4242/v1/mytenant/punchline/executions'

Example response

[
    {
        "id": "wc2Yh3EBnOslsCeb6UAg-1587117889508",
        "timestamp": "2020-04-17T10:04:55.320Z",
        "duration": 229,
        "tenant": "mytenant",
        "event": "spark.application.start"
    }
]

GET /v1/{tenant}/punchline/executions/{id}/events

List all events of a specific execution.

Request

Path variables

Field Description
tenant Tenant name
id Execution ID

Response

Body

Field Description
[].timestamp Event timestamp
[].tenant Tenant name
[].event Execution event

Status

Status Code Reason
200 successful operation
404 Execution not found

Example request

curl --location --request GET \
'http://localhost:4242/v1/mytenant/punchline/executions/wc2Yh3EBnOslsCeb6UAg-1587117889508/events'

Example response

[
    {
        "timestamp": "2020-04-17T10:04:58.247Z",
        "tenant": "mytenant",
        "event": "spark.application.end"
    },
    {
        "timestamp": "2020-04-17T10:04:58.241Z",
        "tenant": "mytenant",
        "event": "spark.application.end"
    },
    {
        "timestamp": "2020-04-17T10:04:58.217Z",
        "tenant": "mytenant",
        "event": "spark.application.end"
    },
    ...
]

GET /v1/{tenant}/punchline/executions/{id}/output

Get output of a specific execution.

Request

Path variables

Field Description
tenant Tenant name
id Execution ID

Response

Status

Status Code Reason
200 successful operation
404 Execution not found

Example request

curl --location --request GET \
'http://localhost:4242/v1/mytenant/punchline/executions/wc2Yh3EBnOslsCeb6UAg-1587117889508/output'

GET /v1/{tenant}/punchline/executions/{id}

Get executions for a specific punchline.

Request

Path variables

Field Description
tenant Tenant name
id Punchline ID

Response

Body

Field Description
[].id Execution ID
[].timestamp Last event timestamp
[].duration Execution duration
[].tenant Tenant name
[].event Execution last event

Status

Status Code Reason
200 successful operation
404 Punchline not found

Example request

curl --location --request GET 'http://localhost:4242/v1/mytenant/punchline/executions/wc2Yh3EBnOslsCeb6UAg'

Example Response

[
    {
        "id": "wc2Yh3EBnOslsCeb6UAg-1587117889508",
        "timestamp": "2020-04-17T10:04:58.247Z",
        "duration": 3156,
        "tenant": "mytenant",
        "event": "spark.application.end"
    }
]

POST /v1/{tenant}/punchline/save

Save a punchline

Request

Path variables

Field Description
tenant Tenant name

Form

Field Description
file Punchline file

Response

Status

Status Code Reason
200 successful operation

Example request

curl --location --request POST 'http://localhost:4242/v1/mytenant/punchline/save' \
--form 'file=@/data/punchlines/dataset_generator.pml'

GET /v1/{tenant}/punchline/scan/analytics

Get all nodes available for Spark punchline.

Request

Path variables

Field Description
tenant Tenant name

Response

Status

Status Code Reason
200 successful operation

Example request

curl --location --request GET 'http://localhost:4242/v1/mytenant/punchline/scan/analytics'

GET /v1/{tenant}/punchline/scan/pyspark

Get all nodes available for Pyspark punchline.

Request

Path variables

Field Description
tenant Tenant name

Response

Status

Status Code Reason
200 successful operation

Example request

curl --location --request GET 'http://localhost:4242/v1/mytenant/punchline/scan/pyspark'

GET /v1/{tenant}/punchline/scan/storm

Get all nodes available for Storm punchline.

Request

Path variables

Field Description
tenant Tenant name

Response

Status

Status Code Reason
200 successful operation

Example request

curl --location --request GET 'http://localhost:4242/v1/mytenant/punchline/scan/storm'

DELETE /v1/{tenant}/punchline/scan/reset

Force the reset of the scanner cache

Request

Path variables

Field Description
tenant Tenant name

Response

Status

Status Code Reason
200 successful operation

Example request

curl --location --request GET 'http://localhost:4242/v1/mytenant/punchline/scan'

GET /v1/{tenant}/punchline/{id}

Get punchline.

Request

Path variables

Field Description
tenant Tenant name
id Punchline id

Response

Body

Field Description
index Elasticsearch index where punchline is stored
id Punchline id
source Punchline file content

Status

Status Code Reason
200 successful operation
404 Punchline not found

Example request

curl --location --request GET 'http://localhost:4242/v1/mytenant/punchline/wc2Yh3EBnOslsCeb6UAg'

Example response

{
    "index": "mytenant-applications",
    "id": "wc2Yh3EBnOslsCeb6UAg",
    "source": {
        "@timestamp": "2020-04-17T10:04:37.537Z",
        "platform_id": "my-unique-platform-id",
        "dag": "{\"runtime_id\":\"wc2Yh3EBnOslsCeb6UAg\",\"metrics\":{\"reporters\":[{\"http_hosts\":[{\"port\":9200,\"host\":\"localhost\"}],\"type\":\"elasticsearch\",\"index_name\":\"mytenant-metrics\"}]},\"job\":[{\"settings\":{\"input_data\":[{\"name\":\"phil\",\"musician\":false,\"age\":21,\"friends\":[\"alice\"]},{\"name\":\"alice\",\"musician\":true,\"age\":23,\"friends\":[\"dimi\"]},{\"name\":\"dimi\",\"musician\":true,\"age\":53,\"friends\":[\"phil\",\"alice\"]}]},\"component\":\"input\",\"publish\":[{\"stream\":\"data\"}],\"description\":\"The batch_input node simply generates some data.\\nYou simply write your data inline, it convert it as Dataset<Row>\",\"type\":\"dataset_generator\"},{\"settings\":{\"truncate\":false},\"component\":\"show\",\"subscribe\":[{\"component\":\"input\",\"stream\":\"data\"}],\"type\":\"show\"}],\"type\":\"spark\",\"tenant\":\"mytenant\"}",
        "tenant": "mytenant"
    }
}

POST /v1/{tenant}/punchline/{id}

Execute a saved punchline

Request

Path variables

Field Description
tenant Tenant name
id Punchline id

Response

Body

Field Description
id Execution generated id

Status

Status Code Reason
200 successful operation
404 Punchline not found

Example request

curl --location --request POST 'http://localhost:4242/v1/mytenant/punchline/wc2Yh3EBnOslsCeb6UAg'

Example response

{
    "id": "wc2Yh3EBnOslsCeb6UAg-1587117889508"
}

DELETE /v1/{tenant}/punchline/{id}

Delete a saved punchline.

Request

Path variables

Field Description
tenant Tenant name
id Punchline id

Response

Status

Status Code Reason
201 successful operation
404 Punchline not found

Example request

curl --location --request DELETE 'http://localhost:4242/v1/mytenant/punchline/wc2Yh3EBnOslsCeb6UAg'

GET /v1/{tenant}/resources/list

List resources

Request

Path variables

Field Description
tenant Tenant name

Parameters

Field Description
all true or false. If false, list the last version of each resource. If true, list all resources versions. Default is false.
artifact Exact artifact name, or a name pattern with wildcard
group Exact group name, or a name pattern with wildcard
type An artifact type. Possible values can be retrieved from the /v1/{tenant}/resources/types route

Response

Body

Field Type Description
group string The group name (ex: org.thales.punch)
artifact string The artifact name
version string The artifact version (ex: 1.3.1)
desciption string A short description of the artifact
displayName string The text to display instead of the artifact name
type string The artifact type (ex: PUNCHLINE)
createdAt string The date at which the artifact was pushed
keywords string[] Keywords for this artifact, as an array of string
links object[] An array of links
links[].name string The text to display for the link
links[].url string The url of the link
readme string The readme of the artifact, using markdown
changelog string[] The changelog for this version, one change per line
maintainers object[] An array representing the maintainers of the artifact
maintainers[].name string The name of the maintainer
maintainers[].email string The email of the maintainer
provider object An object containing information about the publisher of the artifact
provider.name string The name of the publisher
provider.email string The email of the publisher
punchVersionCompatible string[] An array of punch versions with which the artifact is compatible
file string The name of the uploaded artifact build
content string The content of the uploaded artifact build
size number The artifact build size
storage object An object containing information about where the artifact is stored
storage.backend string The backend type (ex: external)
storage.url string The url of the uploaded artifact build
properties object An object with user-defined content

Status

Status Code Reason
200 Resourcce listed

Example request

curl --location --request GET 'http://localhost:4242/v1/mytenant/resources/list?all=true&artifact=myartifact&group=org.thales.punch&type=PUNCHLINE'

Example response

[
  {
    "group": "com.mycompany.artifacts",
    "artifact": "my-artifact",
    "version": "2.5.0",
    "description": "This is a sample artifact",
    "displayName": "My Artifact",
    "type": "PUNCHLINE",
    "createdAt": "2021-05-05 17:30",
    "keywords": [
      "punchline",
      "sample-keyword",
    ],
    "links": [
      {
        "name": "GitHub",
        "url": "https://github.com/mycompany/myartifact"
      }
    ],
    "readme": "Artifact documentation in markdown format",
    "changelog": [
      "feat: add support for foobar",
      "fix: remove a broken item"
    ],
    "maintainers": [
      {
        "name": "John Doe",
        "email": "john.doe@mycompany.com"
      }
    ],
    "provider": {
      "name": "My Company",
      "email": "contact@mycompany.com"
    },
    "punchVersionCompatible": [
      "6.x"
    ],
    "size": 0
  }
]

PUT /v1/{tenant}/resources/upload

Upload a resource

Request

Path variables

Field Description
tenant Tenant name

Form

Field Description
input Input file to upload
manifest Metadata file to upload

Response

Body

The response body for this call has the same format as the one from GET /v1/{tenant}/resources/list (a single object instead of an array)

Status

Status Code Reason
201 Resource uploaded and metadata successfully stored
504 Data storage or metadata storage cannot be reached

Example request

curl --location --request PUT "http://localhost:4242/v1/mytenant/resources/upload" \
--form "input=@\"./my-artifact/build.zip\"" \
--form "manifest=@\"./my-artifact/metadata.yml\""

Example response

See the payload from GET /v1/{tenant}/resources/list

GET /v1/{tenant}/resources/metadata

Get an artifact definition

Request

Path variables

Field Description
tenant Tenant name

Parameters

Field Description
artifact Exact artifact name
group Exact group name
version Specific version to get. If not provided, download the last version of the resource (equals to the highest version number)

Response

Body

The response body for this call has the same format as the one from GET /v1/{tenant}/resources/list (a single object instead of an array)

Status

Status Code Reason
200 Resource downloaded
404 Resource not found

Example request

curl --location --request GET 'http://localhost:4242/v1/mytenant/resources/metadata?artifact=myartifact&group=com.mycompany.artifacts&version=3.5.0'

DELETE /v1/{tenant}/resources/delete

Delete a resource

Request

Path variables

Field Description
tenant Tenant name

Parameters

Field Description
artifact Exact artifact name
group Exact group name
versions Optional. An array of specific versions to delete

Response

Status

Status Code Reason
200 Resource deleted
404 Resource not found

Example request

curl --location --request DELETE 'http://localhost:4242/v1/mytenant/resources/delete?artifact=my-artifact&group=com.mycompany.artifacts'

PUT /v1/{tenant}/resources/copy/{name}

Copy a resource

Request

Path variables

Field Description
tenant Tenant name
name Name of the resource inside the data storage

Form

Field Description
destination Copy location of the resource inside the data storage
version If provided, only copy this version
embedded true or false. If true, embed the resource data content inside the metadata

Response

Body

Field Description
metadata Copied resources details
name Name inside the data storage
message Response description
tenant The tenant of the resource manager

Status

Status Code Reason
201 Resource copied
404 Original resource not found
504 Data storage or metadata storage cannot be reached

Example request

curl --location --request PUT 'http://localhost:4242/v1/mytenant/resources/copy/tests/test.jpg' \
--form 'destination=copies/test.jpg' \
--form 'version=1'

Example response

{
    "metadata": [
        {
            "schema": "V1",
            "@timestamp": "2020-07-06T14:07:21.288Z",
            "size": 4806,
            "name": "copies/test.jpg",
            "storage": {
                "type": "file",
                "url": "/tmp/punchplatform/manager/resources/mytenant/copies/0/test.jpg"
            },
            "version": 0,
            "tenant": "mytenant",
            "properties": {}
        }
    ],
    "name": "tests/test.jpg",
    "message": "Copy resources",
    "tenant": "mytenant"
}

PUT /v1/{tenant}/resources/move/{name}

Copy a resource

Request

Path variables

Field Description
tenant Tenant name
name Name of the resource inside the data storage

Form

Field Description
destination New location of the resource inside the data storage
version If provided, only move this version
embedded true or false. If true, embed the resource data content inside the metadata

Response

Body

Field Description
metadata Moved resources details
name Name inside the data storage
message Response description
tenant The tenant of the resource manager

Status

Status Code Reason
201 Resource copied
404 Resource not found
504 Data storage or metadata storage cannot be reached

Example request

curl --location --request PUT 'http://localhost:4242/v1/mytenant/resources/move/tests/test.jpg' \
--form 'destination=moves/test.jpg' \
--form 'version=1'

Example response

{
    "metadata": [
        {
            "schema": "V1",
            "@timestamp": "2020-07-06T14:16:59.678Z",
            "size": 4806,
            "name": "moves/test.jpg",
            "storage": {
                "type": "file",
                "url": "/tmp/punchplatform/manager/resources/mytenant/moves/0/test.jpg"
            },
            "version": 0,
            "tenant": "mytenant",
            "properties": {}
        }
    ],
    "name": "tests/test.jpg",
    "message": "Move resources",
    "tenant": "mytenant"
}

PUT /v1/{tenant}/resources/register/{name}

Register the metadata of an existing resource

Request

Path variables

Field Description
tenant Tenant name
name Name of the resource inside the data storage

Form

Field Description
version Version of the resource to register
url Exact location of the resource inside the data storage
size Size of the resource to register
properties Custom properties of the resource to register
compression Compression type of the resource, if needed
encoding Encoding type of the data content

Response

Body

Field Description
metadata Registered resource details
name Name inside the data storage
message Response description
tenant The tenant of the resource manager

Status

Status Code Reason
200 Resource registered
404 Resource not found
422 If the resource version already exists

Example request

curl --location --request PUT 'http://localhost:4242/v1/mytenant/resources/register/tests/test.jpg' \
--form 'version=3' \
--form 'url=/tmp/punchplatform/manager/resources/mytenant/tests/0/test.jpg' \
--form 'properties={"description":"hello world, and aliens","test":true}' \
--form 'size=42' \
--form 'compression=gzip' \
--form 'encoding=UTF8'

Example response

{
    "metadata": [
        {
            "schema": "V1",
            "@timestamp": "2020-07-06T14:21:58.716Z",
            "size": 3,
            "name": "tests/test.jpg",
            "storage": {
                "type": "file",
                "encoding": "UTF8",
                "compression": "gzip",
                "url": "/tmp/punchplatform/manager/resources/mytenant/tests/0/nomods.png"
            },
            "version": 3,
            "tenant": "mytenant",
            "properties": {
                "test": true,
                "description": "hello world, and aliens"
            }
        }
    ],
    "name": "tests/test.jpg",
    "message": "Register resource",
    "tenant": "mytenant"
}

GET /v1/{tenant}/documentation/{page}

Display the documentation page content

Request

Path variables

Field Description
tenant Tenant name
page Path of the documentation page

Response

Status

Status Code Reason
200 Page content
404 Page not found

Example request

curl --location --request GET 'http://localhost:4242/v1/documentation/index.html'