Skip to content

Setup and Provisioning

Abstract

Before running application, a few kubernetes objects must be created: namespaces, S3 buckets, etc.. This chapter explain these requirement steps.

Tenants Provisioning

The PConsole punchplatform.properties file contain a kubernetes section where you added your target tenants.

Before using the platform, the administrator needs to first create manually some Kubernetes objects using the following procedure for each defined tenant.

First define a the following environment properties. You must

export ENCODED_SECRET=<BASE_64_ENCODED> 
export SERVICE_ACCOUNT=spark-sa 
export NAMESPACE=mytenant 
export SECRET_NAME=mysecret 
export ROLE_BINDING_NAME=spark-role

Warning

Each identifier in the above commands must exists in the punchplatform.properties file.

Then execute the following:

cat <<EOF | kubectl apply -f -
---
apiVersion: v1
kind: Namespace
metadata:
  name: $NAMESPACE
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: $ROLE_BINDING_NAME
  namespace: $NAMESPACE
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: admin
subjects:
- kind: ServiceAccount
  name: $SERVICE_ACCOUNT
  namespace: $NAMESPACE
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: $SERVICE_ACCOUNT
  namespace: $NAMESPACE
imagePullSecrets:
- name: $SECRET_NAME
---
apiVersion: v1
kind: Secret
metadata:
  name: $SECRET_NAME
  namespace: $NAMESPACE
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: $ENCODED_SECRET
EOF

Channel Configuration

To launch an application, you must simply refer to your kubernetes cluster in the channel channel_structure.yml file as follows:

---
version: '7.0'
applications:
  - name: groupby
    command: pyspark
    cluster: kastcluster
    type: kubernetes
    args: [
      "start",
      "--punchline", "groupby.yaml"
    ]