Python Custom Application¶

Abstract

In addition to the legacy shiva applications, you may want to bring your own Python application. You get the ability to write your own app, add it to punch binaries and customize a shell to launch it as a shiva command.

Warning

You should learn the shiva documentation before going further : Shiva documentation

Requirements¶

To proceed to a Shiva app customization, ensure that :

Shiva is deployed
Punch Operator is deployed
You have an access to these servers
You have read and write permissions on {setups_root} folder on these servers

Info

Check the {setups_root} configuration inside the platform section in punchplatform-deployment.settings documentation

Create a custom Shiva application¶

A custom Shiva application is divided into 3 parts :

A custom binary file application
A custom shell command, using the custom binary
A custom channel structure, using the remote custom shell

Custom binary¶

The custom pex binary must be placed inside your deployer's archives :

├── punch-deployer-6.1.0
│   └── archives
│       └── extlib
|           └── shiva
│               └── my_app
|                   └── my_python_application.pex

The custom pex will be deployed on your platform with Shiva:

punchplatform-deployer.sh --deploy -Kk -t shiva

After the deployment, the custom pex will be placed on the shiva servers inside :

├── punch-binaries-6.1.0
│   └── extlib
│       └── shiva
│           └── my_app
|               └── my_python_application.pex

Info

The punch-binaries-6.1.0 folder is located inside {setups_root}

Info

Refer to this documentation to get more details about how to deploy an external library.

Custom shell¶

The custom shell must be located on an Operator server, inside a channel configuration directory :

├── pp-conf
│   └── tenants
│       └── mytenant
│           └── channels
|               └── mychannel
|                   └── myshell

You may use the following pattern to customize your own launcher :

#!/bin/bash

#
# SET YOUR EXTERNAL RESOURCES HERE
#

SHIVA_EXTLIB="/path/to/punch-binaries-6.1.0/extlib/shiva"
PEX_NAME="my_app.pex"
PEX_PATH="my_path/${PEX_NAME}"

#
# BIN MANAGEMENT
#

pexBin=($(ls -1 ${SHIVA_EXTLIB}/${PEX_PATH} 2> /dev/null ))
if [ -z ${pexBin:-} ] ; then
    echo "FATAL ERROR : Unable to find ${PEX_PATH} in ${SHIVA_EXTLIB}." 1>&2
    exit 1
fi

#
# EXECUTE PEX
#

function start {
    PEX_ROOT=$PUNCHPLATFORM_PEX_CACHE_DIR ${pexBin} $@
}

start $@
exit $!

Make your shell executable by the Shiva daemon user :

cd /path/to/punch-shiva-6.1.0/bin
chmod +x my_shell
chown shiva_user:shiva_user my_shell

Channel Structure¶

The custom channel_structure.json must be located on an Operator server, inside :

├── pp-conf
│   └── tenants
│       └── mytenant
│           └── channels
|               └── mychannel
|                   ├── myshell
|                   └── channel_structure.json

You may use the following pattern to customize your own channel :

{
    "version": "6.0",
    "start_by_tenant" : false,
    "stop_by_tenant" : true,
    "applications": [
        {
            "name": "my_app",
            "runtime" : "shiva",
            "command": "my_shell",
            "agrs": [
                "my_arg1", "my_arg2"
            ],
            "cluster": "my_cluster",
            "shiva_runner_tags": [
                "my_shiva_tag"
            ]
        }
    ]
}

Info

Check more information about channel structure in Channels documentation

Check the validity of your new channel :

channelctl -t my_tenant status

channel:my_channel ................................................... STOPPED

You can now start your custom Shiva application using this new channel !

channelctl -t mytenant start --channel mychannel