Tenant Configuration¶
Tenant Indices¶
The punch relies on elasticsearch not only to store the user business data, but also to centralize metrics and inner logs so as to provide operators with a simple and efficient mean to understand the platform and their application behavior.
As a concrete example we will assume you have a platform with two tenants:
- mytenant : the user applicative tenant. This is where users will setup business or administrative channels. Note that administrative channels refer here to channels in charge of dealing with user data and resources associated to that tenant only.
- platform : on production platforms, a tenant called
platformis defined to deal with platform level monitoring data. Servers cpu, network usage or any other platform level resource is monitored, and in turn have a natural platform level scope.
Tip
Daily weekly or monthly indices are very common. These are depicted hereafter using a mytenant-events-* notation. Using daily indices you will have one index per day mytenant-events-2019.05.09, mytenant-events-2019.05.10 etc..
| Index | Description |
|---|---|
| mytenant-events-* | user data are stored in indices prefixed with the tenant's name. At the end, a date indicates the start of stored events. The 'events' part here is only illustrative but often used for logs. It describes the type of data, and is useful to select only a kind of object through indices names (Kibana index pattern concept) |
| mytenant-metrics-* | Punch applicative components (nodes of punchlines) publish their metrics to daily per-tenant indices. |
| platform-metricbeat-* | These indices contain OS-level resources metrics, captured by metribeat daemons inside all VMs of the Punch platform. Useful for capacity planning and incident troubleshooting. |
| platform-health-* | The punch monitoring service periodically publishes a consolidated platform status document. It is in turn exposed using through the elasticsearch REST api to external supervisors. It provides a complete view of the health status of all platform components (Storm, Spark, Shiva, Elasticsearch, Zookeeper, etc). |
| platform-monitoring-* | In this daily index the punch monitoring service keeps track of the complete monitoring history, individual component by component. |
| mytenant-channels-monitoring-* | The punch channels monitoring service publishes a channel status for each configured channel. |
| platform-operator-logs-* | This monthly index contains the hisotry of the punch operator actions (start and stop actions performed by users through channelctl) This index MUST be kept for at least 1 year to ensure proper channels monitoring |
| platform-logs-* | This index contains the logs of shiva components and of applicatins run inside shiva clusters. This index allows a fine grain monitoring of all platform and shiva scheduled jobs. |
| platform-plan-cursor-* | Provides the ability for a plan to checkpoint the last successful execution. Hence, enabling resilient plans executions. Jobs that are executed are based on an advancing timestamp watermark... |
| mytenant-gateway-logs-* | This index contains all gateway logs relative to a tenant. It contains all the history of requests received by the gateway and the decision of the gateway (access granted or request blocked). |
Important
The precise naming depends on your final configuration, in particular daily monthly or time independant indices are setup in channel configurations. What is however enforced is the tenant prefixe usage, and the platform level indices naming scheme. Overall these naming strategy is also required for enforcing security. The indices names HAVE TO begin by the tenant name, followed by a '-'. This constraint is enforced inside key services, to help isolation between tenants (e.g. : a tenant Elasticsearch housekeeping service CANNOT purge any indice that does not match this prefix pattern ; this avoids that a misconfigured housekeeping service destroys an other tenant's data)