Test policies in depth

This page describes the tests you can write with APD in more detail than Write tests: the test levels, the options of policy tests and attribute connector tests, and how to run the tests and read their results. The examples use the sample project of the APD distribution.

Test levels

APD runs all tests with JUnit 5 and the AlfaExtension. How you set up the extension decides what a test covers:

Level	AlfaExtension	Attribute values
Unit test of a policy	new AlfaExtension().withMainPolicy("consultants.Consultants")	All set in the test with .with(...)
Integration test of a policy with its attribute connectors	new AlfaExtension().withMainPolicy("consultants.Consultants").withAttributeConnectors()	Set in the test, and retrieved by the attribute connectors
System test of the authorization domain	new AlfaExtension().withAttributeConnectors()	Set in the test, and retrieved by the attribute connectors
Unit test of an attribute connector	new AlfaExtension(), with newAttributeTest(...)	Retrieved by the attribute connector

:::warning Be careful with tests that use attribute connectors Attribute connectors retrieve data from systems outside your control, such as directories, databases, and services. That data can change at any time: a user changes role, a record is removed, or a service is unavailable. A test that depends on it can then fail even though your policies are correct, and tests that fail for no clear reason soon stop being trusted.

We recommend that you test your policies and your attribute connectors separately:

• Test your policies with unit tests that set all attribute values in the request.
• Test your attribute connectors on their own, against data you control or a mock of the service.

Check that policies and attribute connectors work together by verifying the authorization domain in a test environment, rather than with automated tests that run on every build. :::

Without withMainPolicy, a test evaluates the main policy of the authorization domain, set with mainpolicy in the alfa { } section of build.gradle. With withMainPolicy, it evaluates another policy, such as one policy of the domain in a unit test.

Unit and integration tests run on the policy engine embedded in APD. System tests can also run against a running ADS, where APD skips the other tests. See Test against a running ADS.

Policy tests

A policy test sends an access request to the policy under test and checks the response:

src/test/java/com/myorg/alfa/MyConsultantPolicyUnitTest.java

package com.myorg.alfa;

import apd.Dictionary;
import com.axiomatics.cr.alfa.test.junit.AlfaExtension;
import com.axiomatics.cr.alfa.test.junit.TestResponse;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import static com.axiomatics.cr.alfa.test.junit.matchers.AlfaMatchers.*;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.CoreMatchers.not;
import static org.hamcrest.MatcherAssert.assertThat;

public class MyConsultantPolicyUnitTest {

    @RegisterExtension
    public AlfaExtension alfa = new AlfaExtension().withMainPolicy("consultants.Consultants");

    @Test
    @DisplayName("REQ-ABAC-2: Should not permit a Consultant to access resource in another location")
    public void shouldNotPermitIfUserAndResourceAreInDifferentLocation() {
        TestResponse result = alfa.newTestRequest()
                .with(Dictionary.USER_ROLE, "consultant")
                .with(Dictionary.USER_LOCATION, "london")
                .with(Dictionary.RESOURCE_LOCATION, "stockholm")
                .evaluate();

        assertThat(result, is(not(permit())));
    }
}

Test request attribute values

Add attribute values to the request with .with(attribute, value). The attribute is a name from attributes.yaml, either as a constant of the generated Dictionary class or as a string, such as .with("user.role", "consultant"). With the constants, the compiler finds a mistyped name. Call .with(...) several times with the same attribute to send several values.

To set the current date or time of a request, see Date and time.

Evaluate the request

.evaluate() sends the request to a Policy Decision Point (PDP) and returns its response, a TestResponse with the decision, advice, and obligations:

• By default, the PDP is the policy engine embedded in APD. It evaluates your policies directly from the project, and writes the visual trace of the evaluation.
• When the environment variable ALFA_TEST_REMOTE_URL is set, .evaluate() sends the request to that URL instead, for example to a running ADS. There is no visual trace then. A running ADS evaluates only the main policy, with its own attribute connectors, so APD skips the tests that need the embedded engine: tests with withMainPolicy for a policy other than the main policy, and attribute connector tests (newAttributeTest(...)). They are reported as skipped. See Test against a running ADS.

Expected decision

Check the decision with the Hamcrest matchers permit(), deny(), notApplicable(), and indeterminate() of AlfaMatchers, combined with is(...) and is(not(...)). See the Hamcrest tutorialOpens in a new tab for more about matchers.

import static com.axiomatics.cr.alfa.test.junit.matchers.AlfaMatchers.*;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.CoreMatchers.not;
import static org.hamcrest.MatcherAssert.assertThat;

TestResponse result = request.evaluate();

// the request is permitted
assertThat(result, is(permit()));

Depending on what the requirement expects, check another decision instead:

// the request is denied
assertThat(result, is(deny()));

// no policy or rule applies to the request
assertThat(result, is(notApplicable()));

// the evaluation failed, for example because an attribute connector could not retrieve a value
assertThat(result, is(indeterminate()));

// any decision other than permit: deny, not applicable, or indeterminate
assertThat(result, is(not(permit())));

is(not(permit())) passes for every decision other than permit. To check that a request is explicitly denied, use is(deny()).

Advice and obligations

Policies can return advice and obligations with a decision. The application that asked for the decision must fulfil the obligations, and can use or ignore the advice. Check them with the matchers withText(...) and withIdAndText(...) of AttributeAssignmentMatcher:

import static com.axiomatics.cr.alfa.test.junit.matchers.AttributeAssignmentMatcher.withIdAndText;
import static com.axiomatics.cr.alfa.test.junit.matchers.AttributeAssignmentMatcher.withText;
import static org.hamcrest.CoreMatchers.hasItem;
import static org.hamcrest.Matchers.empty;
import static org.hamcrest.Matchers.hasSize;

// the advice with the text
assertThat(result.getAdvice(), hasItem(withText("Permit since user is manager")));

// exactly one advice
assertThat(result.getAdvice(), hasSize(1));

// at least one obligation
assertThat(result.getObligations(), is(not(empty())));

// an obligation with the attribute and text
assertThat(result.getObligations(), hasItem(withIdAndText("acme.adviceText", "Obligation from PDP")));

// the obligation with the id myObligation, with the attribute and text
assertThat(result.getObligation("myObligation"), hasItem(withIdAndText("acme.adviceText", "Obligation from PDP")));

getAdvice("id") returns the advice with an id, like getObligation("id") for obligations.

Display names

Give each test a display name with the requirement it verifies, using @DisplayName, as in the example above. The display name is shown in the test report and in your IDE, and is the title of the test's visual trace. Keep the test method name descriptive too: the trace files are named after it.

Attribute connector tests

An attribute connector test calls one attribute connector directly, without policies. Test your attribute connectors during development. PIPs hold data outside your control, so mock them, for example with WireMock, rather than using these tests as acceptance tests.

src/test/java/com/myorg/alfa/MyAttributeconnectorTest.java

package com.myorg.alfa;

import apd.Dictionary;
import com.axiomatics.cr.alfa.test.junit.AlfaExtension;
import com.axiomatics.cr.alfa.test.junit.AttributeConnector;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import java.util.List;

import static org.hamcrest.CoreMatchers.hasItem;
import static org.hamcrest.MatcherAssert.assertThat;

public class MyAttributeconnectorTest {

    @RegisterExtension
    public AlfaExtension alfa = new AlfaExtension();

    @Test
    public void shouldGetRoleConsultantForCecilia() {
        AttributeConnector target = alfa.newAttributeTest("ourConnector");

        List<String> result = target.lookup(Dictionary.USER_ROLE).by(Dictionary.USER_IDENTITY, "cecilia");

        assertThat(result, hasItem("consultant"));
    }
}

Method	Description
newAttributeTest(...)	The file name of the attribute connector in src/authorizationDomain/attributeConnectors, without the extension.
lookup(...)	The output attribute to retrieve: one of the providedAttributes of the attribute connector. For example, .lookup(Dictionary.USER_ROLE) for the user's role. One lookup retrieves one attribute: to test several output attributes of the same attribute connector, call lookup(...).by(...) once for each, for example target.lookup(Dictionary.USER_ROLE).by(...) and target.lookup(Dictionary.USER_LOCATION).by(...).
by(...)	The values of the input attributes that the attribute connector looks up the output attribute by: for each input attribute, the attribute followed by its value, in any order. With one input attribute: .by(Dictionary.USER_IDENTITY, "cecilia"). With two, for example the user's approval limit by the user's identity and the department of a purchase order: .by(Dictionary.USER_IDENTITY, "cecilia", Dictionary.RESOURCE_PURCHASE_ORDER_DEPARTMENT, "InternalSales"). Returns the values of the output attribute as a list.

If an attribute connector uses environment variables, such as URLs and credentials, set them for the tests in the test section of build.gradle. For example, for a variable ${PIP_PASSWORD}:

build.gradle

test {
    environment "PIP_PASSWORD", "examplePassword"
}

Chained attribute connectors

To test chained attribute connectors, use the output of one attribute connector as the input of the next. For the attribute connectors of this example, see Example: HTTP attribute connector chained to a JSON parser.

Chained attribute connector test

@Test
public void shouldGetClearanceForUserAlice() {
    String body = alfa.newAttributeTest("userHttpService")
            .lookup(Dictionary.USER_SERVICE_HTTP_BODY).by(Dictionary.USER_IDENTITY, "alice").get(0);
    String clearance = alfa.newAttributeTest("userHttpServiceParser")
            .lookup(Dictionary.USER_CLEARANCE).by(Dictionary.USER_SERVICE_HTTP_BODY, body).get(0);
    assertThat(clearance, is("classified"));
}

System tests

A system test evaluates the main policy with all attribute connectors, to verify that the authorization domain as a whole meets its requirements. The request contains only what your application sends, and the attribute connectors provide the rest:

src/test/java/com/myorg/alfa/MySystemTest.java

package com.myorg.alfa;

import apd.Dictionary;
import com.axiomatics.cr.alfa.test.junit.AlfaExtension;
import com.axiomatics.cr.alfa.test.junit.TestResponse;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import static com.axiomatics.cr.alfa.test.junit.matchers.AlfaMatchers.permit;
import static com.axiomatics.cr.alfa.test.junit.matchers.AttributeAssignmentMatcher.withText;
import static org.hamcrest.CoreMatchers.hasItem;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.MatcherAssert.assertThat;

public class MySystemTest {

    @RegisterExtension
    public AlfaExtension alfa = new AlfaExtension().withAttributeConnectors();

    @Test
    @DisplayName("REQ-ABAC-3: Should permit manager Martin to access resource 1")
    public void shouldGiveMartinAccessToResource1() {
        TestResponse result = alfa.newTestRequest()
                .with(Dictionary.USER_IDENTITY, "martin")
                .with(Dictionary.RESOURCE_IDENTITY, "1")
                .evaluate();

        assertThat(result, is(permit()));
        assertThat(result.getAdvice(), hasItem(withText("Permit since user is manager")));
    }
}

The attribute connectors need the same environment variables as in their own tests, see Attribute connector tests.

Run the tests and read the results

To	Run
Run all tests	./gradlew test
Run one test class	./gradlew test --tests "com.myorg.alfa.MySystemTest"
Run one test	./gradlew test --tests "com.myorg.alfa.MySystemTest.shouldGiveMartinAccessToResource1"

You can also run tests from your IDE. The test report is in build/reports/tests/test/index.html.

APD writes a visual trace of each policy test to build/alfa/tests/trace/, and prints its location. It shows how the policy was evaluated and which attribute values were used. Visual tracing is enabled by default. To disable it, add withVisualTrace false in the alfa { } section of build.gradle. With visual tracing disabled, .withEvaluationTrace() still writes a trace: on a request for that request only, or on the AlfaExtension for all tests of the class. See Visual tracing.