Skip to content

CRAIG 5.4.0

Below is a summary of the JIRA issues addressed in the CRAIG-5.4.0 release of Punchplatform. For full documentation of the release, a guide to get started, and information about the project, see the Punchplatform project site.

Note about upgrades: Please carefully review the upgrade documentation for this release thoroughly before upgrading your cluster. The upgrade notes discuss any critical information about incompatibilities and breaking changes, performance changes, and any other changes that might impact your production deployment of Punchplatform.

The documentation for the most recent release can be found at https://doc.punchplatform.com.

Release summary

This release brings the update of Elastic and Zookeeper and Ceph COTS as well as a review of the CENTOS deployment

Major changes

Please read Migration Guide from 5.3.0 to 5.4.0

Release notes

Major changes

  • [PP-2487] - Platform monitoring improvements for production
  • [PP-3268] - Merge 'punchplatform-operator-environment-dataanalytics' with defaut operator
  • [PP-3328] - Update platform health/monitoring fields name

Main subject

  • [PP-2851] - Ergonomics and functionality for PML
  • [PP-3224] - Open Distro for standalone
  • [PP-3270] - Job editor for topology

New feature

  • [PP-2875] - Support resilient periodic aggregation computation in CRAIG
  • [PP-3230] - POC standalone with OpenDistro for ES Security plugin
  • [PP-3263] - Basic Auth for Elasticsearch_Spout
  • [PP-3295] - Generate Storm topology from PML
  • [PP-3356] - Draw and execute pyspark with job editor
  • [PP-3326] - add topology start command compatible with new pml generic format

Improvement

  • [PP-3164] - Channel CLI revamp

  • [PP-1891] - finalise multi tenancy monitoring

  • [PP-2925] - Improve ElasticsearchInputNode for less parameters
  • [PP-2927] - Autocompletion ou liste indexes dans ElsaticsearchInputNode
  • [PP-3109] - Integrate PML nodes to use Keras models in pyspark
  • [PP-3175] - PML configuration should work on both java and python
  • [PP-3217] - Improvement/Stabilization for PML JAVA
  • [PP-3240] - Upgrade pyspark core engine
  • [PP-3256] - Upgrade Pyspark to accept basic examples
  • [PP-3259] - Unit tests for pyspark engine
  • [PP-3262] - Rework user experience for extraction by kibana
  • [PP-3267] - POC standalone with OpenDistro sql plugin
  • [PP-3271] - Syslog, kafka, punch and Elastic PML-Storm nodes
  • [PP-3274] - Elasticsearch reporter interface for punch components
  • [PP-3311] - Implements a MITRE use case
  • [PP-3312] - document and deliver production ES templates mappings
  • [PP-3327] - prototype the cyber Thread Intel IOC module
  • [PP-3340] - ElasticSearch REST client API

  • [PP-2898] - Write a dedicated Siddhi documentation

  • [PP-3058] - implement python pml kafka input node
  • [PP-3198] - Improve Getting Started documentation
  • [PP-3265] - Move Shiva 'commands' shells
  • [PP-3272] - improve kibana plugin
  • [PP-3279] - Remove APM server from Standalone
  • [PP-3304] - Templating library for java
  • [PP-3307] - pom improvements
  • [PP-3314] - Performance evaluation when starting many channels
  • [PP-3332] - Provide LTR standard deployment configuration

Bug

  • [PP-3023] - pp-analytics.sh (launching job in client mode should be by default in background)
  • [PP-3150] - Documentation is unclear about dependency of active plugin pages and choice of operator environment
  • [PP-3191] - Kafka spout must replay ONLY failed tuples
  • [PP-3212] - In online puncher, on exception, the exception message/line is not displayed
  • [PP-3241] - Update Zookeeper conf to accept 4 word letters
  • [PP-3242] - Periodically publish monitoring metrics for all technos
  • [PP-3243] - Shiva monitoring: only forward useful fields to ES
  • [PP-3248] - Wrong platform monitoring overview dashboard
  • [PP-3249] - Wrong kafka broker monitoring
  • [PP-3258] - Deployer python PyYaml error message (CVE)
  • [PP-3261] - Objectf storage meta wrong compression values
  • [PP-3266] - Nimbus is Flooded by multiple channel start
  • [PP-3269] - wrong message when channel_structure is hjson and not json
  • [PP-3282] - Fix pp-development.sh install
  • [PP-3286] - Punchctl channel state issue
  • [PP-3292] - Tuple toEscapedJson() wrong format
  • [PP-3306] - Elastic batch input return null column when user selects unknown columns
  • [PP-3322] - punchctl alive session can lead to wrong shiva start command
  • [PP-3329] - Some jar issue on deployer
  • [PP-3330] - Metricbeat index pattern issue on deployer
  • [PP-3336] - Deployer crash when using filebeat with kafka
  • [PP-3344] - ES extraction with logger
  • [PP-3345] - Fix kibana deployment on centos
  • [PP-3346] - No failure/return code for Shiva task (e.g. housekeeper)

Testing

  • [PP-3357] - Craig 5.4.0 - Test campaign