Requirements for deployment and target servers¶

Supported Operating systems¶

The punchplatform deployer is supported on these operating systems (for deployer server, and for targetted deployed servers)

Ubuntu 18.04 or 20.04
Debian 9 (10 not tested but should be compatible)
CentOS 7 or 8
RedHat 7 (8 not tested but should be compatible)

Ceph deployment is currently only provided for Ubuntu 18.04, Debian 9, CentOS 7, RedHAT 8

SSL usage for input/output connector nodes (syslog nodes, lumberjack nodes) is not yet available for Ubuntu 20 and CentOS 8.

Minimum system resources per server¶

No matter which OS you use, the minimum recommendations for all deployers and target servers (or VMs) are as follows (not excluding higher requirements depending on the actual use cases and deployment structure) :

2 CPU
4GB Memory
30 GB Storage. This is typically needed for :
- Operating systems, standard packages binaries, updates... a few GB at least
- logs (/var/log) a few GB
- /home: at least 6 GB for uploading big files through ansible
- Temporary files a few GB
- Punch and dependencies installed binaries: at least 12 GB (to hold 2 full unarchived versions)
- These 30GB are not including room for application data (kafka, elasticsearch, spark, minio...). for application data (/data) a specific additional partition, sized depending on the node, is strongly advised (see partitioning).

Deployer Server Requirements and setup¶

Target Servers Requirements¶

The target servers are the ones installed with binaries and configurations to run the punch components. The punch deployer tool internally leverages ansible.

Make sure you conform to the following requirements for target servers before running deployment tool.

Infrastructure requirements¶

Make sure:

All network interfaces are up and configured.
Storage partitions are writable and mounted except for Ceph intended block device to manage.

System requirements¶

Linux Accounts¶

A service account is used for running punchplatform daemons and pipeline tasks. If this service account is not created on all target servers BEFORE deployment, then they will be created by the deployer (with a kernel-chosen linux numeric id);

The punch deployer uses a single account name, common to all the deployed daemons on all the servers. This services accounts should NOT be sudoer.

If SSL security is to be used, SSL certificates will have to be uploaded (at deployment or certificate change time) in a folder on each server, that the service account is allowed to READ. Depending on the platform security policy, it is advised that these certificates do not belong to the service account, and that only the deployment accounts are allowed to change them.

No user is normally connected using these service accounts, so connection through password should be banned.

The ‘Linux User accounts’ are
Deployment accounts (per capita, or shared accounts, as platform policy decides), used for running the deployer, then connecting using ssh from the deployment machine to all the target VMs. This/these deployment account(s) must be sudoer on all machines. This or these accounts can have passwords and/or ssh-keys for connection, as the platform policy decides.
Platform administrator accounts:

User accounts with sudo rights, for maintainance and incident management low-level actions (services restart, reboots…) They may be the same as the deployment accounts.
Application administrator accounts (= Punch operator accounts)

User accounts without sudo rights. Needed only on “punch operator” environment servers

They are used for tuning/changing the pipelines configuration and correlation/detection enabled rules, and stopping/restarting the pipelines or rule engines.

They can be used for application-level incident management (e.g; replaying data, extracting data, applicative health investigating…)

If they are not created before the punch deployer is launched, the punch deployer will create them. Si if a platform login/pam/LDAP coupling is in action, the operator accounts need to be created before running the punch, otherwise local operator accounts will be created.

Naming resolution¶

Naming resolution must be configured. Short and long hostname are resolved.

When resolving from any target machine the hostname of itself or any other target machine (i.e. the return of 'hostname' command), the result must be the production network interface (as opposed to any other administration or supervision/monitoring dedicated network interface).

Time Synchronisation¶

A full time synchronisation infrastructure like NTP must be configured and running.

Danger

Be careful when you use a virtualization technology. On some of these technologies, the stopped VMs do not get updated time when they are switched on.

The NTP client our agent you are supposed to have configured at system level of the VM will take care of realigning the date-time at boot time of the VM.

BUT some application services may have been started BEFORE your ntp agent had time to realign time, leading to possible application errors or at least misaligned events date in logs.

The solution can be:

Ensure to test on integration platform proper behaviour of the platform components when restarted after having been stopped some hours.
If needed, enrich the systemd configuration in charge of running your NTP agent, so that it starts BEFORE punch systemd-managed services.

System packages repository¶

Standard repositories mirror of the chosen operating system have to be provided by the infrastructure to compensate (probably in some dedicated server or VM inside the platform local network) to compensate for 'offline' nature of most production platform.

The the target servers have to be configured to use these local repositories, before trying to run punchplatform deployer.

To test the correct configuration, we recommend to update all servers before the punch deployment.
For CentOS deployment, EPEL standard extensions repository must also be available and enabled.

system language¶

System language must be configured to en_US.UTF-8.

localectl | grep LANG
     System Locale: LANG=en_US.utf8

If this is not the case, change it :

sudo localectl set-locale LANG=en_US.utf8

Swapping¶

Swapping is a performance killer. On production servers, swapping should be deployed only as a lifesaver for the VM, not for everyday use

sudo sysctl vm.swappiness should report 0. If this is not the case, add the setting in /etc/sysctl This requests that swap is not used until the physical RAM is exhausted
the platform monitoring (outside of punchplatform deployment) should be configured to display alerts whenever the used swap is not 0

Partitioning¶

It is strongly advised to have separate filesystem partitions for separate usage

the OS (/ filesystem)
the temporary folder (/tmp)
the logs folder (/var/log)
the application data folder (usually /data on punch platforms)
the archiving data folder (usually a nfs mount point e.g. /mnt/archiving)

Specifically, if Zookeeper is colocated with other components (like Kafka or Elasticsearch), it should have its own, small dedicated partition (/data/zookeeper) to avoid corruption if an other application fills the data disk

The punch framework application binaries are taking up to 7-10 GB when fully uncompressed on a target server. It is therefore advised either to have a separate /opt partition, sized for at least 2 releases (15GB) or to store the punchplatform framework binaries inside the big /data partition (putting binaries in /data/opt). This is controlled by the platform.setups_root setting of deployment settings.

Elasticsearch requirements¶

The execution prerequisites are disclosed in Elasticsearch public documentation; the current section highlights some specificities that are often not identified during server/OS setup and may cause deployment problems

the temporary directory for Elasticsearch (by default /tmp), cannot be a partition mounted with noexec option, otherwise Elasticsearch will fail its bootstrap checks with an error message in its log file and therefore the service will not start.

This kind of option is usually used in hardened environment to prevent executable command injection through an other linux account via this shared temporary folder.

To address this issue, you can use the temporary_directory optional deployment setting in the Elasticsearch deployment section to select an alternative path, that must be a temporary folder in a partition that matches the prerequisite (such as a data older dedicated for elasticsearch user, and protected by linux access rights to prevent other accounts to inject executable files in there.)

Elasticsearch by default requires that its process memory may be locked to prevent swapping. The target servers (or virtual servers) must be able to deliver this feature (i.e. no specific hardening) should prevent Elasticsearch from requesting memory lock from the kernel. Elasticsearch checks this at startup time during its 'bootstrap checks' and fail if no/not enough memory could be locked.

Storm requirements¶

The temporary directory for Storm (by default /tmp), cannot be a partition mounted with noexec option, otherwise Storm will fail its bootstrap checks with an error message in its log file and therefore the service will not start.

To address this issue, you can use the temporary_directory optional deployment setting in the Storm deployment section to select an alternative path, that must be a temporary folder in a partition that matches the prerequisite.