Skip to content

Security

Abstract

The punch integrates Opendistro, the security plugin from amazon for Elasticsearch from Amazon. It allows you to protect your data, and to define various role-based access control rules so as to expose your data to your users the way you need.

Overview

In this guide, you will learn to :
1. Equip your punch with the security plugins 2. Secure your punch with SSL and access control without any plain text credentials in configuration files
3. Secure your cluster with authentication and access control

Installation

First, make sure that Elasticsearch and Kibana are not running.

punchplatform-standalone.sh --stop

Then enable the security plugin. This can be performed on your existing standalone.

cd $PUNCHPLATFORM_CONF_DIR/..
./install.sh --with-opendistro-security
source ./activate.sh

and restart your platform

punchplatform-standalone --start

Once ElasticSearch and Kibana are running, open a web browser and go to http://localhost:5601. You should hit this window :

image

Congratulations ! You have secured ElasticSearch and Kibana with authentication and access control for your cluster ! It is as simple as that.

Walkthrough

This presentation will go through the Kibana interfaces, but keep in mind that all the configuration steps described below can be performed through configuration files.

First, log on to Kibana with user admin and password admin.

Security

When you are connected, click on the Security tab. You should see this:

image

Let us describe this security interface :

Category Description
Action Groups Provides the creation of actions (a set of permissions) to grant access to a data type or API
Roles Provides the creation of roles to define a scope for action groups on clusters, indices, documents or fields. It also provides the creation of tenants
Users Provides the creation of user accounts
Role Mappings Provides the mappings between the created roles and the created users
Authentication and Authorization Provides information about the authentication and authorization mechanisms used for your ElasticSearch instance

The most important thing to understand is role management. Go to Roles, then click on all_access role to edit it.

image

The sections that can be edited are :

Section Description
Overview A summary of all users, backend roles and host permissions affected to the roles
Cluster Permissions Provides cluster-wide permissions by assigning action groups to the cluster
Index Permissions Provides index permissions by assigning action groups to indices and document types
DLS/FLS Provides document and field permissions by writing an ElasticSearch query that affect an index. Also provides a way to include, exclude or anonymize fields
Tenants Provides tenant permissions for the current role, based on read/write access

As you can see, the security plugin is focusing on indices and documents access control. You can now manage actions and roles, create users and bind them all together with mappings inside the Security tab to grant permissions to your data.

Tenants

Click on the Tenants tab. You should see this:

image

From Opendistro security plugin, this feature allows a user to self-manage the access controls to its own index patterns, visualizations and dashboards and to protect them from other users. Each tenant can be understood as a workspace where a user's monitoring creations are stored.
For example, if a visualization is created with the private tenant selected, this visualization will not be viewable from any other selected tenant or user.

The default tenants are :

tenant name Description
Global Accessible from any other tenant or user
Private Accessible from this tenant and user only
{custom name} Attached to a role, this tenant is accessible for any other user who shares the same role

For example, the tenant admin_tenant can be accessed from any other user who has the all_access role.

Influence on PunchPlatform components

Since a basic authentication is now needed to connect to Elasticsearch, additional settings must be placed in every components that needs to read or write data in Elasticsearch indices.

Topologies

Additional settings for ElasticSearch spouts and bolts are needed. Please check credential setting : * Elasticsearch Input * Elasticsearch Output

Data analytics nodes

In pml files, add inside elastic_settings :

{
  ...
    elastic_settings : {
      es.net.http.auth.user: "user"
      es.net.http.auth.pass: "pass"
    },
  ...
}

SSL

The Punchplatfrom standalone can be installed with a full SSL configuration providing encrypted connections for :

  • Elasticsearch
  • Kibana
  • Metricbeat
  • Rest API Gateway

The main advantage to this feature is that the services needing an Elasticsearch connection can be configured without credentials

# stop your standalone and install full security conf
punchplatform-standalone.sh --stop
./install.sh --with-security

Restart your standalone and verify the installation :

punchplatform-standalone.sh --start
export CERTS_DIR=$PUNCHPLATFORM_CONF_DIR/resources/ssl/certs/punch_user

curl -v GET https://localhost:9200/_cat/indices \
    --cacert $CERTS_DIR/punch-user-ca.pem \ 
    --cert $CERTS_DIR/punch-user-client-cert.pem \
    --key $CERTS_DIR/punch-user-client-key.pem


< HTTP/1.1 200 OK
< content-type: text/plain; charset=UTF-8
< content-length: 570
< 
green open .kibana_1                            8_JmsSMRT_S7XpInvUlmVA 1 0   0 0   230b   230b
green open .opendistro_security                 eYMYebqGRfimgT5ZGwHz3A 1 0   5 0 25.2kb 25.2kb
green open platform-metricbeat-7.8.0-2020.04.27 pyR2dcm6SuKwZN6xvyZZaw 1 0 348 0  712kb  712kb
green open security-auditlog-2020.04.27         jCb8bOPYSQWbcgMDLc_azQ 1 0   2 0 23.9kb 23.9kb

Info

Where are the credentials you may ask, right ? Well, Punchplatform standalone provides a configuration that uses certificates to authenticate services. The punch user certificates can be userd as admin certificates, and other are provcided for different kind of services in $PUNCHPLATFORM_CONF_DIR/resources/ssl/certs

Warning

Some services still work with basic credentials which need to be forwarded to Elasticsearch. It is the case for Kibana for example : the user's credentials are forwarded to Elasticsearch, but the connection is still encrypted with TLS to protect the critical information

At this point, the Punchplatform Rest API Gateway is also working with enciphered connection to Elasticsearch.

For Elasticsearch forwarding (endpoint: <tenant>/es/<cluster>), the user MUST still provide its credentials, which are forwarded with its request. It allows Opendistro Security to verify each request's original client. The connexion remains encrypted with TLS :

curl -v GET https://localhost:4242/v1/mytenant/es/es_seach/_cat/indices \ 
    -u admin:admin \
    --cacert $PUNCHPLATFORM_CONF_DIR/resources/ssl/certs/punch_user/punch-user-ca.pem

< HTTP/1.1 200 
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: 0
< Strict-Transport-Security: max-age=31536000 ; includeSubDomains
< X-Frame-Options: DENY
< Content-Length: 582
< Date: Tue, 11 Feb 2020 10:49:15 GM

green open .kibana_1                            8_JmsSMRT_S7XpInvUlmVA 1 0   0 0   230b   230b
green open .opendistro_security                 eYMYebqGRfimgT5ZGwHz3A 1 0   5 0 25.2kb 25.2kb
green open platform-metricbeat-7.8.0-2020.04.27 pyR2dcm6SuKwZN6xvyZZaw 1 0 348 0  712kb  712kb
green open security-auditlog-2020.04.27         jCb8bOPYSQWbcgMDLc_azQ 1 0   2 0 23.9kb 23.9kb

Elasticsearch reporters in channels must be configured with SSL. In this case, you can configure them with certificates location. No credentials are needed in this case :

{
    "reporters" : [
      {
          "type" : "elasticsearch",
          "cluster_name" : "es_search",
          "ssl": true,
          "ssl_private_key": "/path/to/punch-user-client-key.pem",
          "ssl_certificate": "/path/to/punch-user-client-cert.pem",
          "ssl_trusted_certificate": "/path/to/punch-user-ca.pem"
      }
    ]
}

Warning

In a production environment, you should use dedicated client key files for each specific service (eg: gateway, reporter, ...)

Add key files as a new role's user in Opendistro

Considering you already have a client key and a client certificate, you have to edit $PUNCHPLATFORM_CONF_DIR/../external/elasticsearch-7.8.0/plugins/opendistro_security/securityconfig with the common name of your certificate :

my_role:
  reserved: false
  backend_roles:
    - "my_role"
  description: "My specific access role"
  users:
    - "my-cert-CN-value"

Then update your cluster security information with admin certs :

punchplatform-elasticsearch.sh --start
cd $PUNCHPLATFORM_CONF_DIR/../external/elasticsearch-7.8.0/plugins/opendistro_security/tools
chmod +x securityadmin.sh
./securityadmin.sh -cd ../securityconfig/ -icl -nhnv \
    -cacert ../../../config/elasticsearch-admin-ca.pem \
    -cert ../../../config/elasticsearch-admin-cert.pem \
    -key ../../../config/elasticsearch-admin-key.pem

Opendistro Demonstration

The punch ships with a small demo so that you can play right away with some data, users and roles.

Installation

First, make sure that ElasticSearch and Kibana are running, then execute :

# disable ssl configuration 
./install.sh --with-opendistro-security
# install demo
$PUNCHPLATFORM_CONF_DIR/samples/opendistro/install_demo.sh

When it is DONE, go to your running Kibana instance and connect to the admin account with user admin, password admin.
Click on Stack Management > Index Patterns > Create index pattern. You can see two new indices : ppsecurity_q1 and ppsecurity_q2.

Demonstration Use case

Consider a monitoring project named PPSecurity.

The monitoring activity is lasting for several month so monitoring data was split into two quarters : Q1 and Q2.
The data contain connection information about four projects (Project A, B, C and D) in two working zones (Zone 1 and 2), as follows :

Zone 1 Zone 2
Project A and B Project C and D

On kibana, through roles, role mappings and user definitions, you can gather the following information :

role permissions role mapping (users)
head_manager Grant access to all data of PPSecurity Sam
project_manager Grant access to all documents in Zone 1 Rosie
engineer Can only monitor data for index ppsecurity_q1 Merry

Thus, Rosie, who is a project manager, can access to any data in all indices that concerns all the projects inside the first zone only.

Practice

As head manager

Logout from admin and connect to Sam account with username sam and password samsam. You can notice that you have no rights to manage the security tab from this user anymore, since you are not currently logged as the admin user.

Go to Tenants tab and select Private tenant.

Go to Management tab and define a new index pattern with pp*. Choose fields.event_timestamp as time filter then click on Create index pattern. Now, by clicking on Discover tab and by selecting your new index pattern, you can display 4 documents with various information about zones and projects that are all the data available for the project PPSecurity, since head_manager has no restrictions on PPSecurity indices.

To push the experience a little further, go to Tenant tab and select another tenant then go back to Discover tab. Your previous index named pp* is not accessible anymore. You can restore it by selecting the tenant you were in when you created the index pattern. This is how tenants work.

As engineer

Let us now check Merry's permissions by connecting to the admin user and go to Security > Roles > engineer > Index Permissions

image

Because of indices_all action group permission assigned to index ppsecurity_q1, Merry will be able to monitor all data inside this index but he also won't be able to access to ppsecurity_q2's data at all.

Now log in merry account with password merry.

Go to Stack Management tab then click on Index Patterns. As explained above, the only index visible for Merry is the one we just precised in security settings for his role. You can now create an index pattern by defining ppsecurity_q1 then any visualizations or dashboards from its data, but Merry will never access to ppsecurity_q2 index.

Warning

Be careful that your index pattern does include only authorized indexes. For instance, if you create the index pattern _pp_, it will include the index ppsecurity_q2*, which is forbidden. Therefore, you will get an error message and will not be able to query this index pattern in Discover.

Feel free to change the index pattern for permissions or even action groups to test all the possibilities of Index Permissions security settings.

As project manager

Let us check Rosie's permissions by connecting to the admin user and go to Security > Roles > project_manager > Index Permissions > Document Level Security Query. Check the ElasticSearch query written for index pattern pp* :

{
  "bool": {
    "must": {
      "match": {
        "fields.cloud.zone_type": "Zone1"
      }
    }
  }
}

Because of this query, Rosie can access to any document containing "Zone1" as value for the field "fields.cloud.zone_type" inside any index beginning with "pp".

Now log in rosie account with password rosie.

Go to Management tab and create a new index pattern with pp*, then display documents in Discover tab. As explained above, the only documents visible for Rosie are those with the value Zone1 for zone_type field, without taking into account the project names.

Another possibility is to change this query to grant to Rosie the access to all documents containing ProjectA or ProjectC as value of the field fields.user.project :

{
  "bool": {
    "must": {
      "bool": {
        "should": [
          {
            "match": {
              "fields.user.project": "ProjectA"
            }
          },
          {
            "match": {
              "fields.user.project": "ProjectC"
            }
          }
        ]
      }
    }
  }
}

Rosie can now access to all document matching this query. The permissions are immediately updated on the Rosie's account.

Feel free to change the index pattern or even the entire query to test all the possibilities of DLS/FLS security settings.