HOWTO activate logging
Why do that¶
To debug or understand better the behavior of a Punchline using Spark/Pyspark runtime.
You need a punch-standalone installed with spark. The easiest way to
work with Spark is to launch the job in foreground mode by using the
punchlinectl command. For example if you have a job
application.hjson file, use the following command
punchlinectl start --punchline application.hjson -v
What to do¶
Configure the Spark log4j.properties¶
Spark use log4j. It is located in
# You will have to update those files $PUNCHPLATFORM_LOG4J_CONF_DIR/log4j2-sparkctl.properties $PUNCHPLATFORM_LOG4J_CONF_DIR/log4j2-sparkctl.xml
By default the delivered log4j.properties is configured only with ERROR level so as to limit standard output to the most relevant spark outputs.
- org.apache.spark : the Spark loggers
- org.thales.punch : the various punchplatform loggers