Skip to content

CRAIG 5.1.1

Below is a summary of the JIRA issues addressed in the CRAIG-5.1.1 release of Punchplatform. For full documentation of the release, a guide to get started, and information about the project, see the Punchplatform project site.

Note about upgrades: Please carefully review the upgrade documentation for this release thoroughly before upgrading your cluster. The upgrade notes discuss any critical information about incompatibilities and breaking changes, performance changes, and any other changes that might impact your production deployment of Punchplatform.

The documentation for the most recent release can be found at https://doc.punchplatform.com.

Release summary

This release brings major fixes to version 5.1.0. It also brings deployment improvements and provides MapD and Azure bolts.

Important changes

  • Storm upgraded from 1.1.1. to 1.1.3
  • If you are using Shiva for scheduling some tenants tasks (services), you have to update your tenants/<tenant>/services/<service>/service_structure.json configuration

Please read Migration Guide from 5.1.0 to 5.1.1

Release notes

Main subjects

  • [PP-2486] - Brad to Craig Migration
  • [PP-2762] - Mapd Integration
  • [PP-2763] - Azure Integration

New feature

  • [PP-2663] - New spout for Azure Blob storage
  • [PP-2737] - Make MapD a third party bolt
  • [PP-2781] - Parser F5 waf

Improvement

  • [PP-2719] - Add a troubleshooting sheet on Locales (LC_)
  • [PP-2721] - Remove deployment parameter "remote_ansible_user"
  • [PP-2730] - Improve the handling of large json files
  • [PP-2731] - Put back evaluation period on the standalone
  • [PP-2741] - Allow providing specific API hosts:ports lists for Elasticsearch monitoring
  • [PP-2748] - shiva task must accept more than one parameter
  • [PP-2750] - Updated documentation 'HOWTO filter data with a punch?'
  • [PP-2758] - remove elastalert to punch
  • [PP-2759] - update pp.properties and pp-deployment.settings released, beta and legacy content
  • [PP-2769] - Improve FileSpout with zip file handling and directory consuming pattern
  • [PP-2790] - Archiving indexation records in ES lack ts and some metadata
  • [PP-2794] - Ssh parsing
  • [PP-2801] - update sudo parser
  • [PP-2802] - firewalld parser
  • [PP-2803] - create the unit test for varonis
  • [PP-2806] - Watchdog on storm tuple ack is needed
  • [PP-2814] - Upgraded storm to 1.1.3 for performance/ack leak risk - [STORM-2231]
  • [PP-2737] - improve third party topologies support

Bug

  • [PP-2657] - Exception not logged in ElasticsearchBolt causing difficult diagnosis
  • [PP-2689] - Unflat tuple doesn't work when nestedSeparator settings is not a dot
  • [PP-2691] - FileBolt does not fail tuples when ES indexing error occurs
  • [PP-2697] - metric storm.tuple.ack.m1_rate is 0 for transactional kafka spout
  • [PP-2707] - Mapping exception are not indexed (always fail)
  • [PP-2709] - undefined service_manager when using -t to deploy
  • [PP-2712] - Index OutOfBound Exception in Kafka consumer when 5 partitions or more
  • [PP-2729] - leak in the underlying tuple boon json parser
  • [PP-2733] - Shiva deployment broken with supervisord variant (ubuntu 14)
  • [PP-2735] - Objects storage indexing lacks pool and cluster information
  • [PP-2745] - Storm supervisors are reported 'red' because their name in properties do not match their declared host in cluster
  • [PP-2749] - Shiva tasks logs are always going to localhost:9200
  • [PP-2751] - unable to install libapr1 on centos7
  • [PP-2753] - Unable to deploy kibana_plugin on centos7
  • [PP-2755] - IllegalStageException raised by Kafka on Seek() inside fail() of PartitionImpl
  • [PP-2764] - topo light spout null pointer exception
  • [PP-2767] - ES indexation Rejection for overload should NOT conduct to reindexation as error documents
  • [PP-2768] - tuple should fail if indexation of errors is activated and fails
  • [PP-2770] - id_rsa.pub is required at deployment
  • [PP-2771] - User create with wrong shell in ubuntu 18.04
  • [PP-2780] - ansible variable conflict between beats
  • [PP-2791] - Dead broker is not detected by Platform monitoring if already dead when shiva leader (re)starts
  • [PP-2808] - Mandatory options to deploy shiva never used
  • [PP-2810] - "complete batch not found" error log in FileBolt
  • [PP-2811] - BatchKafaSpout does not replay failed batches and loses offset
  • [PP-2813] - Storm Performance (ElasticsearchBolt) and tuples loss (ack leak) risk due to Storm 1.1.1 bug

Internal Task

  • [PP-2548] - OpenSSL deployment
  • [PP-2649] - Add SSL Elasticsearch client to Elasticsearch bolt
  • [PP-2673] - Configuration commands overview
  • [PP-2696] - Kibana Plugin must execute a getconf before manage channels
  • [PP-2711] - Move pp-core/punchplatform-plugins to pp-gui
  • [PP-2766] - Operations/Platform_Deployment sections simpler and better organised
  • [PP-2800] - Create a contribute section in documentation
  • [PP-2785] - Adapt parser to parse all given format

Testing

  • [PP-2795] - Test Campaign 5.1.1