Skip to content

Automatic Validation

This page aims to list the current existing tests or validation process, and try to list what they are actually testing.

Robot tests

Overview

These tests are using the python Robot Framework. They basically run a series of commands and mark the result as success or failure depending on a condition. Since we can provide custom libraries in Python, these tests can be quite customizable. They also have a timeout concept, failing the test after some time. Another great advantage is the report it's providing, making it easy to check what succeeded or failed.

Packaging

Tests are packaged with the standalone in the validation tenant.

They can be triggered using the punch-robot-test commands. To make it more simple, an activate.sh is available in the robot dir of validation tenant :

cd $PUNCHPLATFORM_CONF_DIR/tenants/validation/resources/robots
source activate.sh

A robot test can be run according to the file hierarchy or tags, allowing to only run a specific set of tests :

# Run all tests
punch-robot-test .

# Run specific test (ex : archiving)
punch-robot-test archiving

Of course, these tests should be able to run on a deployed punch.

Standalone validate

To run a full test suite on a fresh standalone, you can simply unzip your standalone archive and run ./validate.sh script :

unzip punch-standalone-*.zip
cd punch-standalone*
./validate.sh

This script :

  • Install your standalone
  • Start your standalone
  • Run all robot tests
  • Put the report in Reports Page.

Debug a robot test

If a Robot test fails, you can debug it using the robot report. Reports are available in the robot_output directory : 

xdg-open $ROBOT_OUTPUT_DIR/report.html
From there, you can check your suite or even directly your tests by selecting All tests at the bottom of the page.

You also have your test logs in the robot_stdout directory (use the $ROBOT_STDOUT_DIR to have its location).

Unit Tests

Unit tests using JUnits are meant to test the code. The most code coverage we have, the safer we are. A report is generated during the building process and included in the Reports Page. An organized list of these tests can be found there.