Track 3 Spark Java Node Development

Abstract

This track explains how you can code your own python custom node.

Checkout the $PUNCHPLATFORM_CONF_DIR/training/aim/track3 folder. All files referenced in this chapter are located in that folder. First read carefully the README.md file.

Dependency Management

Read the Depedency Management Guide to understand the issues at stakes.

Development

You can use the IDE (or text editor) of your choice.

Use the punchpkg tool to package and deploy your nodes on your local standalone platform. Refer to the PunchPkg Section

Example

Working directory

cd $PUNCHPLATFORM_CONF_DIR/training/aim/track3

Prerequisite

A standalone installed: https://punchplatform.com

Coding/deploying your custom node

Follow our development guide

Try it out !

Use maven to package the compiled bytecodes in a jar

# build the project
mvn clean install

# a jar with dependencies is generated in $(pwd)/target
# install it with punchpkg
punchpkg spark install target/punch-spark-node-starter-kit-*-jar-with-dependencies.jar

# test the newly installed node
punchlinectl start -p input_node_example.yaml