Skip to content

punchplatform-analytics.sh

NAME

punchplatform-analytics.sh - commands to launch PML analytic plans and jobs

SYNOPSIS

Command to launch PML job:

  • punchplatform-analytics.sh --job PATH [ARGUMENTS]

Command to launch PML plan:

  • punchplatform-analytics.sh --start TENANT/PLAN [ARGUMENTS]

DESCRIPTION

punchplatform-analytics.sh provides the high level command to launch plans and jobs.

OPTIONS

  • --start TENANT/PLAN [ARGUMENTS]

    • Tenant friendly plan launcher. It starts a plan using the "job.template" and "plan.json" files located in the folder$PUNCHPLATFORM_CONF_DIR/tenants/<tenant>/plans/<plan>respectively as template and plan. The spark cluster is selected from "analytics_spark_cluster" field in tenant configuration json "$PUNCHPLATFORM_CONF_DIR/tenants/etc/conf.json".
  • --job PATH [--spark-cluster CLUSTER_NAME | --spark-master MASTER_URL) [ARGUMENTS]

    • Starts a job from the configuration file.

      • --spark-cluster <cluster_name>

        • Submit the jobs/plans to the 'cluster_name' spark_cluster defined in "punchplatform.properties".
      • --spark-master <master_url>

        • Provide a valid spark master url to submit jobs, see Spark URL documentation for available formats. Setting this option overrides --spark-cluster option.

ARGUMENTS

  • --deploy-mode [client | cluster]

  • --json

    • Set the submission output to JSON only. The returned JSON contains the runtime_id job runtime identifier to track the corresponding spark application status.
  • --date DATE

    • Set an execution date. In plan mode (with --start and --plan commands), only one job is launched with a templating corresponding to this date. Its format must be ISO 8601 with offset from UTC ("2011-12-03T10:15:30+01:00").
  • --job PATH

    • Set the job configuration file path.
  • --json-errors

    • With this option, in case of exception in validating the PML, the related errors will be reported to stderr as json Exception objects, for machine handling purpose
  • --verbose-data | -vd

    • With this option, the input and output data will be output to stdout for each node of the PML job
  • --verbose | -v

    • Activates all available details (includes -vd) : underlying java command line, full java stack trace in case of error, input/output data dump of PML nodes.

Optional option but can be used to override "punchplatform.properties" configuration:

  • --spark-home PATH

    • Set the Spark installation location.
  • --client-jar PATH

    • Set the client jar location in client machine.
  • --client-jar-main <main>

    • Set the client jar main.
  • --app-resource-jar PATH

    • Set the app resource jar location in Spark machine(s).
  • --app-resource-main <main>

    • Set the app resource jar main.
  • --additional-jars

    • Set app resource additional jar locations in Spark machine(s).

EXAMPLE

To start the plan "detection_suspect_url" of tenant "analytics":

1
punchplatform-analytics.sh --start analytics/detection_suspect_url

To start the plan "detection_suspect_url" of tenant "analytics" without Spark cluster:

1
punchplatform-analytics.sh --deploy-mode client --spark-master local[*] --start analytics/detection_suspect_url

To start one job from the plan "detection_suspect_url" of tenant "analytics":

1
punchplatform-analytics.sh --date 2011-12-03T10:15:30+01:00 --start analytics/detection_suspect_url 

To start a job

1
punchplatform-analytics.sh --deploy-mode client --spark-master local[*] --job ./your_job.json