Skip to content

CRAIG 5.1.0

Below is a summary of the JIRA issues addressed in the CRAIG-5.1.0 release of Punchplatform. For full documentation of the release, a guide to get started, and information about the project, see the Punchplatform project site.

Note about upgrades: Please carefully review the upgrade documentation for this release thoroughly before upgrading your cluster. The upgrade notes discuss any critical information about incompatibilities and breaking changes, performance changes, and any other changes that might impact your production deployment of Punchplatform.

The documentation for the most recent release can be found at https://doc.punchplatform.com.

Release summary

This release brings major fixes to version 5.0.0. It also brings deployment improvements and provides Kibana plugin, Archiving and Ceph.

Important changes

  • Elasticsearch/Kibana upgraded to 6.4.0
  • Spark upgraded to 6.3.2

Please read Migration Guide from 5.0.0 to 5.1.0

Release notes

Main subjects

  • [PP-1756] - Punchplatform Kibana Plugin
  • [PP-2010] - Archiving refactor (Configuration, indexation)
  • [PP-2199] - Centos ceph deployment

New feature

  • [PP-2428] - develop the log injector kafka consumer feature with the new kafka consumer
  • [PP-1514] - Add centos7 support for deployment

Improvement

  • [PP-979] - static punch tuple must be considered fully empty
  • [PP-991] - Tuple Missing Implementation
  • [PP-2414] - update audit conf for python3
  • [PP-2472] - create an HOWTO to create a ceph pool
  • [PP-2502] - audit python3 support trouble
  • [PP-2546] - Convert documentation to markdown
  • [PP-2662] - multiple output tuples for PML Punch Stage

Bug

  • [PP-986] - syslog spout prints wrong value in local_host
  • [PP-1554] - pygregator tests fail to be launched
  • [PP-2183] - bug doc : modifier la doc sur le worker child opts storm dans la doc de déploiement qui n'est pas claire
  • [PP-2220] - An Elasticsearch-Modsecurity-Kibana instance lets requests pass instead of blocking them
  • [PP-2262] - SSL vulnerabilities on LumberJack/Syslog
  • [PP-2266] - light topology emits double tuples
  • [PP-2267] - Inconsistent choice of Zk cluster for platform configuration storage
  • [PP-2277] - one node datamanagement : storm admin cluster url missing in punchplatform properties
  • [PP-2278] - one node : data analytics : log4j-core jar missing
  • [PP-2324] - java api must not create empty zookeeper nodes by side effect
  • [PP-2329] - spark audit conf must be improved
  • [PP-2331] - punchplatform-puncher.sh does not set PP_CONF_DIR/......./patterns as default grok patterns path
  • [PP-2345] - punchplatform-log-injector.sh --punchlet doesn't work
  • [PP-2367] - besoin de debug sur service de monitoring kafka (dans shiva)
  • [PP-2376] - audit de conf platform.platform_es_cluster mandatory alors qu'il ne devrait pas
  • [PP-2385] - Missing SublimText resources on standalone
  • [PP-2386] - punchplatform-conf-edit.sh bug
  • [PP-2398] - OOM handler does not always fire
  • [PP-2399] - punchplatform-log-injector.sh generates first uuid null
  • [PP-2401] - Wrong return code when "exit leadership, quitting"
  • [PP-2402] - image size in official documentation
  • [PP-2403] - punch convertor operator does not deal correctly with arrays
  • [PP-2407] - Audit configuration tool raise an exception in particular conditions
  • [PP-2409] - zookeeper locks do not work well
  • [PP-2415] - standalone kibana resources upload issue
  • [PP-2420] - kafka bolt exit on failure parameter not documented
  • [PP-2426] - wrong elastic component version in craig-sprint-14 on all 1-nodes
  • [PP-2441] - elasticsearch logger does not reopen the client after an error
  • [PP-2468] - kafka offset commit can rewind partitions to earliest if there is no traffic
  • [PP-2500] - kv operator fails when given wrong input string
  • [PP-2506] - Suspicion of bug in transactional KafkaSpout
  • [PP-2509] - NullPointer exception in ElasticsearchBolt on Elasticsearch error
  • [PP-2518] - PP Plugin display only documentation
  • [PP-2521] - PP Plugin issues kibana 6.4
  • [PP-2522] - PP Plugin can't load standalone exemple with %%PUNCHLET%%
  • [PP-2523] - disable ansible gathering facts caching for deployer
  • [PP-2530] - getconf limitation
  • [PP-2531] - Non-emitted errors in punch bolt when publishing non-received _ppf_id
  • [PP-2534] - separate unit test with environment validation and production configuration
  • [PP-2541] - Empty log tuple emitted by CRAIG LumberjackSpout at each BRAD _ppf_metric reception
  • [PP-2562] - punchplatform-objects-archiving.sh extract-scope does not work in INTO variant
  • [PP-2648] - punchplatform-push-es-templates.sh silently fails if incorrect template
  • [PP-2655] - ArchiveSpout is not replaying (pseudo-randomly : deterministic depending on archivec ontent)
  • [PP-2677] - elasticsearch spout cannot read from remote clusters
  • [PP-2679] - activate.sh not working in standalone
  • [PP-2680] - file spout missing path.regex documentation
  • [PP-2687] - Faulty 'Unknown tenant ' when starting a topology
  • [PP-2690] - tuple assignements fail to handle moving sub-tuple use cases
  • [PP-2699] - Archiving housekeeping JVM does not stop
  • [PP-1230] - GROK operator Can't compile Punch code properly - wrong error message
  • [PP-2432] - Make the KafkaBolt accept a field based load balancing strategy

Internal Task

  • [PP-1751] - Check that /data (for ES nodes, especially) does not need to be included in /etc/updatedb.conf exclusions
  • [PP-1818] - Shiva must forward metrics to Elasticsearch/Kafka
  • [PP-2330] - python2.7 is required on the deployer only to execute gethostbyname

  • [PP-2322] - Shiva should not create Zk kafka node if not exists

  • [PP-2337] - Allow conf audit for "platform.platform_id" only
  • [PP-2693] - Release note "Technique"
  • [PP-2358] - 5.0.0 blocking bug for release

Testing

  • [PP-2549] - Test campaign for CRAIG 5.1.0