The goal of this document is to recommend Punch deployment practices. This reference architecture conveys a general architecture that should be adapted to accommodate the specific needs of each implementation.
This design is the recommended architecture for production environments, as it provides flexibility and resilience.
It is a major architecture recommendation that the Punch Kibana and operator servers, processing servers are separate from data storage servers.
Processing servers, in charge of running punchlne, can introduce unpredictable resource utilization. Separating Vault and Consul allows each to have a system that can be sized appropriately in terms of CPU, memory and disk. Consul is a memory intensive application and so separating it to its own resources is advantageous to prevent resource contention or starvation. Dedicating a Consul cluster as a Vault storage backend is also advantageous as this allows the Consul cluster to be upgraded only as required to improve Vault storage backend functionality. This is likely to be much less frequently than a Consul cluster that is also used for service discovery and service segmentation.
Vault to Consul backend connectivity is over HTTP and should be secured with TLS as well as a Consul token to provide encryption of all traffic. See the Vault Deployment Guide for more information. As the Consul cluster for Vault storage may be used in addition and separate to a Consul cluster for service discovery, it is recommended that the storage Consul process be run on non-default ports so that it does not conflict with other Consul functionality. Setting the Consul storage cluster to run on 7xxx ports and using this as the storage port in the Vault configuration will achieve this. It is also recommended that Consul be run using TLS.
The Punch is designed to handle different failure scenarios that have different probabilities. When deploying a Vault cluster, the failure tolerance that you require should be considered and designed for. In OSS Vault the recommended number of instances is 3 in a cluster as any more would have limited value. In Vault Enterprise the recommended number is also 3 in a cluster, but more can be used if they were performance replicas to help with workload. The Consul cluster is from one to seven instances. This should be an odd number to allow for leadership elections to always resolve. It is recommended that the Consul cluster is at least five instances that are dedicated to performing backend storage functions for the Vault cluster only.