Skip to content

IntelliJ Debugging Tips

Debugging spark punchlines

Classpath

File > Project Structure > Modules then choose punchplatform-analytics-client.

Select tab Dependencies, then add (+) > Jars or directories and select where Spark jars are available.

<your-standalone>/punch-standalone-5.2.0/external/spark-2.4.0-bin-hadoop2.7/jars.

Run and Debug a punchline

Run > Edit Configuration > + > Application

Choose a name for example: PML foreground. - Choose the main class : org.thales.punch.ml.client.ClientMain

VM options

Here is a typical set ov VM options

-Dpunchplatform.configurationFile=<your-standalone>/punch-standalone-5.2.0/conf/punchplatform.properties \
-Dlog4j.configurationFile=<your-standalone>/punch-standalone-5.2.0/external/punch-operator-dataanalytics-5.2.0/bin/log4j2.xml \
-Dlogfile.name=punchlinctl \
-Dpunchplatform.log.dir=/tmp/logs

Program arguments

--verbose \
--app-resource-jar <your-standalone>/punch-standalone-5.2.0/external/spark-2.4.0-bin-hadoop2.7/punchplatform/analytics/job/app_resource/punchplatform-analytics-job-5.2.0.jar \
--app-resource-main org.thales.punch.ml.job.SparkMain \
--additional-jars <your-standalone>/punch-standalone-5.2.0/external/spark-2.4.0-bin-hadoop2.7/punchplatform/analytics/job/app_resource/punchplatform-analytics-job-5.2.0.jar:/home/romain/Documents/punchplatform/craig/standalone/latest/punch-standalone-5.2.0/external/spark-2.4.0-bin-hadoop2.7/punchplatform/analytics/job/additional_jars/* \
--deploy-mode foreground \
--name pml \
--tenant mytenant \
--punchline <your-pml-path> \
--verbose-data
Environment variables

Add your PUNCHPLATFORM environment variables

PUNCHPLATFORM_CONF_URL=localhost:2181/punchplatform-primary
#PUNCHPLATFORM_CONF_DIR=<your-path>/punch-standalone-5.2.0/conf
PUNCHPLATFORM_LOG_DIR=/tmp/log
Use classpath of module

  • Select punchplatform-analytics-client

  • Check Include dependencies with "Provided" scope

Debugging the puncher

Run an application

Run > Edit Configuration > + > Application

Name

Choose a name for example: puncher.

Main Class

com.thales.services.cloudomc.punchplatform.punch.core.Puncher

VM options

-Dpunchplatform.configurationFile=<your-standalone>/punch-standalone-5.2.0/conf/punchplatform.properties \
-Dlog4j.configurationFile=<your-standalone>/punch-standalone-5.2.0/external/punch-operator-dataanalytics-5.2.0/bin/log4j2.xml \
-Dlogfile.name=punchplatform-puncher \
-Dpunchplatform.log.dir=/tmp/logs

Program arguments

-gi  "Jan 2 06:25:43 mailserver14 postfix/\"cleanup[21403]: EF25A72965: message-id=<toto@mailserver14.example.com>" \
-gp "%{GREEDYDATA:syslog_message}"
Environment variables

Add your PUNCHPLATFORM environment variables

PUNCHPLATFORM_CONF_URL=localhost:2181/punchplatform-primary
#PUNCHPLATFORM_CONF_DIR=<your-path>/punch-standalone-5.2.0/conf
PUNCHPLATFORM_LOG_DIR=/tmp/log

Use classpath of module

  • Select punchplatform-analytics-client

  • Check Include dependencies with "Provided" scope