CAMP

Amplify your testing with more configurations!

View the Project on GitHub STAMP-project/camp

CAMP generate

CAMP generates new configurations given a description of what can be varied. We use here a simple made-up example to explain how to generate all configurations, or only the subset that covers all possible variations. We then explain how to define variables and constraints. We also give a few shell commands to visualize the generated configurations.

CAMP Model

An “Awesome” Example

We describe here how to use CAMP to vary the testing configurations of a made-up service called Awesome. This example includes the following five components:

  1. The component tests provides the Tests service and requires the Awesome one. It includes a threads variable, whose values are range from 10 to 50, and the distance between two subsequent sample shall be at most 40.

  2. The awesome component is the system under tests. It provides the Awesome services and requires the DB service that both components mysql and postgres provide. To run, it also requires the Python feature.

  3. The mysql component provides the DB service and includes a variable to model its two versions, namely v5 and v8.

  4. The postgres component also provides the DB services.

  5. The python component provides the Python feature.

Here is the YAML snippet that captures it:

goals:
  running: [ Tests ]

components:
  tests:
    provides_services: [ Tests ]
    requires_services: [ Awesome ]
    variables:
      threads:
        values:
          range: [10, 50]
          coverage: 40
  awesome:
    provides_services: [ Awesome ]
    requires_services: [ DB ]
    requires_features: [ Python ]
  mysql:
    provides_services: [ DB ]
    variables:
      version:
        values: [ v5, v8 ]
  postgres:
    provides_services: [ DB ]
  python:
    provides_features: [ Python ]

We can ask CAMP to generate all the possible configurations, as follows:

$ camp generate -d . --all

The figure below shows what configurations CAMP generates.

Awesome configurations

Variations Coverage

CAMP can enumerate all the possible configurations, but it can also generate a smaller subset that covers all possible variations at least one (i.e., components and variable values).

$ camp generate --coverage -d .

In the previous example, CAMP generates only the three following configurations:

awesome configurations

Features vs. Services

The CAMP model distinguishes between services and features. Services are endpoints exposed on the network whereas as features are capabilities available within the same container.

CAMP only connects component instances when one provides at least one of the services the other requires. The same hold for features, the only difference is that a component can only use the feature of a single component.

Variables

CAMP let you defined the variables that may vary in each component. When CAMP instantiate these components, it will try to find values for each variable defined in related the component. Variables may be of several types:

Constraints

You can also add constraints using the Ozepy DSL in order to rule out some configurations. This is done in the constraints section, placed at the top level, as follows:

goals: ...
components: ...
constraints:
    - CInstance.forall(ci.configuration.forall(val, val.value == 0))

Here we define a constraint that force all the component instance, to have 0 as the value for all their variable. We must refer to the CAMP metamodel to write correct constraints.

Visualization

CAMP also generate Graphviz representation (file named configuration.dot) of each configuration it generates. Provided that graphviz is installed, you can convert a configuration using:

$ dot -Tpng configuration.dot -o configuration.png

Alternatively, you can convert all configurations using the following on-liner:

$ find . -name "*.dot" | xargs -I file dot -Tpng file -o file.png

You may also want to gather all these configuration views into a single picture. Provided the imagemagick tools are available on your machine, you can use the following command:

$ find . -name "*.png" \
   | tr '\n' ' ' \
   | montage  -label '%d/%f' @- -geometry 300x300 configurations.png

The CAMP Metamodel

The figure below illustrates the CAMP model. The green classes will be generated by the constraint solver, whereas the yellow ones are specified in the YAML file.

CAMP metamodel