Skip to content

Artifacts Server

The artifact server allows you to manage parsers, pex, or zip archives and use them in particular in punchlines.

Graphical User Interface

If you go to the root url (for example http://localhost:4245/) you can view the graphical user interface to manage artifacts.

Command Line Interface

You can use resourcectl command to manage artifacts. for example:

resourcectl --url https://localhost:4245 list

REST Interface

Artifact server is a REST-LIKE (not RESTFull) API to manage artifacts.

Get artifact types

Request

# List all available types (punch-parsers, additional-spark-jar,...)
curl -X GET "http://localhost:4245/v1/artifacts/types"

Response

[
  "file",
  "punch-parsers",
  "additional-spark-jar",
  "additional-punch-jar",
  "additional-pyspark-pex",
  "punchline",
  "patch"
]

Upload artifact

Request

# Upload one artifact
curl -X POST "http://localhost:4245/v1/artifacts/upload" -F artifact=@artifacts.zip -F override=true

# Upload many artifacts
curl -X POST "http://localhost:4245/v1/artifacts/upload" -F artifact=@artifacts_1.zip -F artifact=@artifacts_2.zip

# PARAMS:
#artifact: binary file (can be use multiple times)
#override: force update data and metadata

Response

// List of uploaded metadata
[
  {
    // metadata content
  }
]

Register external artifact

Request

# Upload artifacts with metadata
curl -X POST "http://localhost:4245/v1/artifacts/register" -F metadata=@metadata.yml -F url="http://my-external-service/artifact.zip" -F override=false

# PARAMS:
#metadata: metadata file (see metadata section bellow)
#url: external data url (where artifact is stored)
#override: override existing metadata
{
  // metadata content
}

Get artifact metadata

Request

# Get metadata for a specific artifact
curl -X GET "http://localhost:4245/v1/artifacts/metadata?\
    group=org.thales.punch\
    &artifact=printroot\
    &version=1.0.0"

# PARAMS:
# group: artifact group
# artifact: artifact name
# version: artifact version, default latest

Response

{
  "group": "org.thales.punch",
  "artifact": "printroot",
  "version": "1.0.0",
  "description": "This package is an punchlet example which print root document",
  "displayName": "Punchlet Print root",
  "type": "punch-parsers",
  "createdAt": "2021-05-05 17:30",
  "logo": "https://foobar.com/foobar.jpg",
  "keywords": [
    "Punchlet",
    "Root",
    "Print"
  ],
  "links": [
    {
      "name": "Punch Print Root documentation",
      "url": "https://doc.punchplatform.com/Reference_Guide/Punchlines/Spark/Nodes/Input/Elastic_Input.html"
    }
  ],
  "readme": "Package documentation in markdown format\nContent added here will be rendered on Artifact Hub\n",
  "install": "Add dependency in your punchline (stormline)\n```yaml\ndependencies:\n  - punch-parsers:org.thales.punch:printroot:1.0.0\n```\n",
  "changelog": [
    "Initial print root version"
  ],
  "maintainers": [
    {
      "name": "Thales Punch",
      "email": "contact@punchplatform.com"
    }
  ],
  "provider": {
    "name": "Thales Punch",
    "email": "contact@punchplatform.com"
  },
  "punchVersionCompatible": [
    "7.0.1"
  ],
  "originalFileName": "printroot-1.0.0.zip",
  "filename": "printroot-1.0.0.zip",
  "files": [
    {
      "path": "printroot-1.0.0/org/thales/punch/printroot/printroot.punch",
      "size": -1
    }
  ],
  "size": 1293,
  "storage": {
    "backend": "FILE_SYSTEM",
    "url": "org/thales/punch/printroot/1.0.0/printroot-1.0.0.zip"
  },
  "fullVersionedArtifactName": "org.thales.punch:printroot:1.0.0",
  "fullArtifactName": "org.thales.punch:printroot",
  "semVersion": {
    "preReleaseVersion": "",
    "buildMetadata": "",
    "patchVersion": 0,
    "normalVersion": "1.0.0",
    "minorVersion": 0,
    "majorVersion": 1
  }
}

List artifacts

Request

# List all registered artifacts
curl -X GET "http://localhost:4245/v1/artifacts/list"

# List all registered artifacts with filters
curl -X GET "http://localhost:4245/v1/artifacts/list?\
    group=org.thales.punch\
    &artifact=printroot\
    &type=PUNCH_PARSERS\
    &all=false"

# PARAMS:
# group: artifact group
# artifact: artifact name
# type: artifact type (see /types to get available types)
# all: display all versions (default false only latest returned)

Response

[
  {
    "group": "org.thales.punch",
    "artifact": "printroot",
    "version": "1.0.0",
    "description": "This package is an punchlet example which print root document",
    "displayName": "Punchlet Print root",
    "type": "punch-parsers",
    "createdAt": "2021-05-05 17:30",
    "logo": "https://foobar.com/foobar.jpg",
    "keywords": [
      "Punchlet",
      "Root",
      "Print"
    ],
    "links": [
      {
        "name": "Punch Print Root documentation",
        "url": "https://doc.punchplatform.com/Reference_Guide/Punchlines/Spark/Nodes/Input/Elastic_Input.html"
      }
    ],
    "readme": "Package documentation in markdown format\nContent added here will be rendered on Artifact Hub\n",
    "install": "Add dependency in your punchline (stormline)\n```yaml\ndependencies:\n  - punch-parsers:org.thales.punch:printroot:1.0.0\n```\n",
    "changelog": [
      "Initial print root version"
    ],
    "maintainers": [
      {
        "name": "Thales Punch",
        "email": "contact@punchplatform.com"
      }
    ],
    "provider": {
      "name": "Thales Punch",
      "email": "contact@punchplatform.com"
    },
    "punchVersionCompatible": [
      "7.0.1"
    ],
    "originalFileName": "printroot-1.0.0.zip",
    "filename": "printroot-1.0.0.zip",
    "files": [
      {
        "path": "printroot-1.0.0/org/thales/punch/printroot/printroot.punch",
        "size": -1
      }
    ],
    "size": 1293,
    "storage": {
      "backend": "FILE_SYSTEM",
      "url": "org/thales/punch/printroot/1.0.0/printroot-1.0.0.zip"
    },
    "fullVersionedArtifactName": "org.thales.punch:printroot:1.0.0",
    "fullArtifactName": "org.thales.punch:printroot",
    "semVersion": {
      "preReleaseVersion": "",
      "buildMetadata": "",
      "patchVersion": 0,
      "normalVersion": "1.0.0",
      "minorVersion": 0,
      "majorVersion": 1
    }
  }
]

Delete artifact and metadata

Request

# Upload artifacts with metadata
curl -X DELETE "http://localhost:4245/v1/artifacts/?group=org.thales.punch&artifact=printroot"

# PARAMS:
#group: artifact group
#artifact: artifact name

Response

# 200 OK

Update artifact and metadata

# update => will be deprecated if flavor to PUT /
POST /update

Download resource

Request

# Download artifact
curl -X GET "http://localhost:4245/v1/artifacts?group=org.thales.punch&artifact=printroot&version=1.0.0" -o binary.zip

# PARAMS:
#group: artifact group
#artifact: artifact name
#version: version 

Response

 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1293    0  1293    0     0   126k      0 --:--:-- --:--:-- --:--:--  126k

Metadata

Minimal metadata

Bellow, it is an example of minimal metadata to use to store artifact.

NOTE: With this minimal file, graphical user interface (UI) will not pretty display artifacts.
Read full metadata section bellow to improve user experience.

type: punch-parsers
group: org.thales.punch
artifact: printroot-ex2
version: 1.0.0

Full metadata

It is preferable to use full metadata file when manage artifacts to provides useful indications about your artifacts. Example bellow describe all available fields

# MANDATORY
type: punch-parsers
group: org.thales.punch
artifact: printroot
version: 1.0.0

# MANDATORY FOR GRAPHICAL USER INTERFACE
displayName: Artiract pretty name
description: Short description about your artifacts, will appair in result list
readme: |
  You can provides information about artifact in the readme section.
  Note this section is multiline and markdown friendly while you keep indentation.
  Content added here will be rendered on Artifact server on artifact page detail
install: |
  Provides installation instruction inside this section.
  Note this section is multiline and markdown friendly while you keep indentation.
  For example why do not specify punchline `dependencies` section like this:  
  ---
  dependencies:
    - punch-parsers:org.thales.punch:printroot:1.0.0  
  ---

### OPTIONAL FOR GRAPHICAL USER INTERFACE 
logo: https://foobar.com/foobar.jpg
keywords:
  - Punchlet
  - Print Root
  - Parser
links:
  - name: Punch Print Root documentation
    url: https://doc.punchplatform.com
changelog: |
  Describe changes between version here.  
maintainers:
  - name: Thales Punch
    email: contact@punchplatform.com
    url: https://punchplatform.com
provider:
  name: Thales Punch
  email: contact@punchplatform.com
punchVersionCompatible:
  - 7.0.1

Punch parser artifact

Helping section about artifact for punch-parsers type.

punch-parsers artifact type has particular workflow when punchline starts. Punchline will extract artifacts with this type, and expect normalized structure. Keep in mind creating parsers with this following structure.

Example Considering you want to create parsers punch-parsers:org.thales.punch:common-parsers:1.0.0 which contains only one punchlet.
Artifact (ZIP archive) look like this:

common-parsers-1.0.0/
|_ org/
  |_ thales/
    |_ punch/
      |_ common-parsers/
        |_ myparser.punch