5.0.0 to 5.1.0¶
This document explains what configuration changes MUST be performed during a PunchPlatform update from version 5.0.0 to 5.1.0
Breaking change in configuration:¶
zookeeper_admin_cluster_name
is mandatory
clusters.<clusterId>.settings_by_type.client_type.readonly
is also mandatory
bolt_settings encoding mandatory¶
when starting a channel 'bolts.bolt_settings.encoding' is missing
.
Update version¶
update 5.0.0 to 5.1.0-SNAPSHOT update elastic components to 6.4.0 update spark to 6.3.2
prevents error¶
To prevents error "Unrecognized field admin_es_shiva_log_index_prefix"
when staring a channel:
1 2 3 4 5 6 7 8 9 10 | com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "admin_es_shiva_log_index_prefix" (class org.thales.punch.platform.api.pojo.TenantEtcConfPojo), not marked as ignorable (11 known properties: "admin_es_health_index_prefix", "admin_es_command_index_prefix", "archiving_pools", "admin_es_cluster", "admin_es_analytics_job_index_prefix", "admin_zk_cluster", "analytics_spark_cluster", "admin_es_analytics_plan_index_prefix", "admin_es_metrics_index_prefix", "max_validity_of_monitoring_results_in_minutes", "elasticsearch_housekeeping"]) at [Source: {"admin_zk_cluster":"common","admin_es_cluster":"es_search","analytics_spark_cluster":"spark_main","admin_es_health_index_prefix":"mytenant-health-","elasticsearch_housekeeping":{"clusters_settings":[{"cluster_id":"es_search","actions":[ {"indices_prefix":"mytenant-","older_than_days":7,"type":"delete_indices","indices_naming_time_format":"%Y.%m.%d"} ]}]},"admin_es_shiva_log_index_prefix":"mytenant-shiva-log-","admin_es_command_index_prefix":"mytenant-commands-","admin_es_metrics_index_prefix":"mytenant-metrics-"}; line: 1, column: 392] (through reference chain: org.thales.punch.platform.api.pojo.TenantEtcConfPojo["admin_es_shiva_log_index_prefix"]) at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:62) ... |
delete admin_es_shiva_log_index_prefix
in /etc/conf.json
Known bugs¶
Spark deployment¶
If you try to deploy operator and spark on separate devices, then the deployer will fail with the following error:
1 2 | TASK [spark-setup : unzip punchplatform deployment in spark folder] ********************************************************************************************** fatal: [tdfsocopr01]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: {{punchplatform_deployment_tool}}/archives/{{punchplatform_analytics_deployment_version}}.zip: 'punchplatform_analytics_deployment_version' is undefined\n\nThe error appears to have been in '/data/punchplatform-deployer-X.Y.Z/roles/spark-setup/tasks/main.yml': line 36, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: unzip punchplatform deployment in spark folder\n ^ here\n"} |
To fix the deployer, update the operator group vars templates (punchplatform-deployer-X.Y.Z/inventory_templates/group_vars/punchplatform_operator_servers.group_vars.j2) with the following lines:
1 2 3 4 5 6 7 | {% if punchplatform.spark is defined %} spark_is_configured: True spark_version: {{deployment_settings.spark_version}} punchplatform_analytics_deployment_version: "{{deployment_settings.punchplatform_analytics_deployment_version}}" {% else %} spark_is_configured: False {% endif %} |