5.0.0 to 5.1.0

This document explains what configuration changes MUST be performed during a PunchPlatform update from version 5.0.0 to 5.1.0

Breaking change in configuration:

zookeeper_admin_cluster_name is mandatory

clusters.<clusterId>.settings_by_type.client_type.readonly is also mandatory

bolt_settings encoding mandatory

when starting a channel 'bolts.bolt_settings.encoding' is missing.

Update version

update 5.0.0 to 5.1.0-SNAPSHOT update elastic components to 6.4.0 update spark to 6.3.2

prevents error

To prevents error "Unrecognized field admin_es_shiva_log_index_prefix" when staring a channel:

com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "admin_es_shiva_log_index_prefix" (class org.thales.punch.platform.api.pojo.TenantEtcConfPojo), not marked as ignorable (11 known properties: "admin_es_health_index_prefix", "admin_es_command_index_prefix", "archiving_pools", "admin_es_cluster", "admin_es_analytics_job_index_prefix", "admin_zk_cluster", "analytics_spark_cluster", "admin_es_analytics_plan_index_prefix", "admin_es_metrics_index_prefix", "max_validity_of_monitoring_results_in_minutes", "elasticsearch_housekeeping"])

at [Source: {"admin_zk_cluster":"common","admin_es_cluster":"es_search","analytics_spark_cluster":"spark_main","admin_es_health_index_prefix":"mytenant-health-","elasticsearch_housekeeping":{"clusters_settings":[{"cluster_id":"es_search","actions":[

{"indices_prefix":"mytenant-","older_than_days":7,"type":"delete_indices","indices_naming_time_format":"%Y.%m.%d"}
]}]},"admin_es_shiva_log_index_prefix":"mytenant-shiva-log-","admin_es_command_index_prefix":"mytenant-commands-","admin_es_metrics_index_prefix":"mytenant-metrics-"}; line: 1, column: 392] (through reference chain: org.thales.punch.platform.api.pojo.TenantEtcConfPojo["admin_es_shiva_log_index_prefix"])

at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:62)

...

delete admin_es_shiva_log_index_prefix in /etc/conf.json

Known bugs

Spark deployment

If you try to deploy operator and spark on separate devices, then the deployer will fail with the following error:

TASK [spark-setup : unzip punchplatform deployment in spark folder] **********************************************************************************************
fatal: [tdfsocopr01]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: {{punchplatform_deployment_tool}}/archives/{{punchplatform_analytics_deployment_version}}.zip: 'punchplatform_analytics_deployment_version' is undefined\n\nThe error appears to have been in '/data/punchplatform-deployer-X.Y.Z/roles/spark-setup/tasks/main.yml': line 36, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: unzip punchplatform deployment in spark folder\n  ^ here\n"}

To fix the deployer, update the operator group vars templates (punchplatform-deployer-X.Y.Z/inventory_templates/group_vars/punchplatform_operator_servers.group_vars.j2) with the following lines:

{% if punchplatform.spark is defined %}
spark_is_configured: True
spark_version: {{deployment_settings.spark_version}}
punchplatform_analytics_deployment_version: "{{deployment_settings.punchplatform_analytics_deployment_version}}"
{% else %}
spark_is_configured: False
{% endif %}