Pact tests: what about state data?

If you're new to Consumer-driven Contracts and Pact, you can get the basics from this previous article: Why you should use Consumer-Driven Contracts for Microservice integration tests

One of the challenges of setting up Pacts between Teams is to get Pacts at the right level so that every Team can work faster, with the confidence that making a breaking change will be detected before it makes it to production.

On the one hand, if the contracts are too basic (e.g. describing happy paths only), some cases will be missed. On the other, very constraining contracts will cause spurious Pact-verification failures on non-breaking changes, which will grind the whole process to a halt.

Contracts can be too strict in different ways. One of them is when Contract tests are being confused with Functional tests. I won’t go into detail on this in this post, there was a recent discussion on the Pact Gitter about it, which resulted in a page in the Pact documentation: The difference between contract tests and functional tests

Getting Consumer and Provider to agree on State

Now, assuming that Contracts are written at the right level (verifying requests and response structures, as opposed to testing provider business logic), the next risk is to get tied down to strict test data definitions. Most services will rely on some persisted state of some kind (usually in a database) which will control what data will be returned in responses.

When defining the contract, Consumers will have different expectations of what state the Provider will be in before making requests. Luckily, Pact covers this with the Provider state part of the integration definition. Provider state is very similar to the Given part of the BDD-style scenario definition.

For example, we can have these two user-lookup scenarios:

Given that a user exists with the uuid abc123  
When making a GET request to /users/abc123  
Then response is 200 with body { "uuid": "abc123", "username": "tom" }  
Given that there is no user with the uuid abc123  
When making a GET request to /users/abc123  
Then response is 404  

Because these two scenarios assume different states for the Provider, they need to be specified as part of the contract; otherwise, the Provider won’t know what kind of setup is needed before replaying the interaction.

The consumer tests code for these two interactions will looks like this:

PactFragment pactFragment = ConsumerPactBuilder  
  .consumer("login-service")
  .hasPactWith("users-service")
  .given("a user exists with the uuid abc123")
  .uponReceiving("request for an existing user")
  .path("/users/abc123")
  .method("GET")
  .willRespondWith()
  .status(200)
  .body("{ \"uuid\": \"abc123\", \"username\": \"tom\" }")
  .toFragment();

VerificationResult result = pactFragment.runConsumer(MockProviderConfig$.MODULE$.createDefault(), config -> {  
  UserServiceConsumer serviceConsumer = new UserServiceConsumer(config.url());
  User user = serviceConsumer.getUser("abc123");
  assertThat(user.getUUID()).isEqualTo("abc123");
  assertThat(user.getUsername()).isEqualTo("tom");
});

if (result instanceof PactError) {  
  fail("Pact verification failed", ((PactError) result).error());
}

assertThat(ConsumerPactTest.PACT_VERIFIED).isEqualTo(result);  
PactFragment pactFragment = ConsumerPactBuilder  
  .consumer("login-service")
  .hasPactWith("users-service")
  .given("there is no user with the uuid abc123")
  .uponReceiving("request for an unknown user")
  .path("/users/abc123")
  .method("GET")
  .willRespondWith()
  .status(404)
  .toFragment();

VerificationResult result = pactFragment.runConsumer(MockProviderConfig$.MODULE$.createDefault(), config -> {  
  UserServiceConsumer serviceConsumer = new UserServiceConsumer(config.url());
  try {
    serviceConsumer.getUser("abc123");
    failBecauseExceptionWasNotThrown(UserNotFoundException.class);
  } catch(UserNotFoundException e) {
    // expected
  }
});

if (result instanceof PactError) {  
  fail("Pact verification failed", ((PactError) result).error());
}

assertThat(ConsumerPactTest.PACT_VERIFIED).isEqualTo(result);  

The resulting pact will then contain both interactions, with their associated provider state:

...
"interactions": [
    {
      "description": "request for an existing user",
      "request": {
        "method": "GET",
        "path": "/users/abc123"
      },
      "response": {
        "status": 200,
        "body": {
          "uuid": "abc123",
          "username": "tom"
        }
      },
      "providerState": "a user exists with the uuid abc123"
    },
    {
      "description": "request for an unknown user",
      "request": {
        "method": "GET",
        "path": "/users/abc123"
      },
      "response": {
        "status": 404
      },
      "providerState": "there is no user with the uuid abc123"
    }
  ]
...

Finally, when the Provider needs to verify this Pact, it will be able to rely on the Provider state to provision test data before replaying each interaction.

The JUnit integration library makes this very easy using annotations in conjunction with the PactRunner.

For our example, this is the only bit of additional test code the Provider needs:

@State("a user exists with the uuid abc123")
public void stateUserExists() {  
  dbHelper.runQuery("INSERT INTO users VALUES('abc123', 'tom')");
}

@State("there is no user exists with the uuid abc123")
public void emptyState() {  
  // Nothing to do
}

The great thing about specifying the state in plain common language is that the Consumer does not need to worry about how the data is injected into the Provider states.

One thing to be careful about is the potential proliferations of states, which the Provider will end up having to manage. If the lines of communications are open between teams, it doesn’t end being a big challenge though.

Improving the contracts tolerance with Matchers

Defining Provider states is a great first step to aligning the Consumer and Provider for each interaction, unfortunately there are some things Consumers and Provider will never be able to agree on in advance (and that’s ok!). Automatically generated fields are a common example of this, and most Pacts dealing with creation operations are likely to hit this issue.

For example, this user creation scenario:

Given that username anna is not already taken  
When making a POST request to /users with body { username: anna }  
Then response is 200 with body {  
"uuid": "CA8BDwYICQ4JDAcOBAAKDg",
"username": "anna"
}

Since the Provider is automatically generating UUIDs, it will never be able match the expected response body exactly. But does the Consumer really care about the exact value of the UUID anyway?

By defining Matchers, a Consumer can be more relaxed about how it defines the expected response. In this specific case, a response can be expected to contain a uuid field that is a string—if the value differs from the expected response but the type matches, the Provider verification will pass.

What it looks like for the Consumer is a slightly different Pact Fragment definition:

PactFragment pactFragment = ConsumerPactBuilder  
  .consumer("registration-service")
  .hasPactWith("users-service")
  .given("username anna is not already taken")
  .uponReceiving("request to create a new user with a free username")
  .path("/users")
  .method("POST")
  .body("{\"username\": \"anna\"}")
  .willRespondWith()
  .status(200)
  .body(new PactDslJsonBody()
    .stringType("uuid", "CA8BDwYICQ4JDAcOBAAKDg")
    .stringValue("username", "anna"))
  .toFragment();

The important piece here is:

new PactDslJsonBody()  
  .stringType("uuid", "CA8BDwYICQ4JDAcOBAAKDg")
  .stringValue("username", "anna")

It specifies that the expected response contains:

  • a UUID field which is a string (with an example value)
  • a username field which should strictly be "anna" (we really care about the value of this one!)

The generated Pact will have an extra field to include the matching rules:

{
  "description": "request to create a new user with a free username",
  "request": {
    "method": "POST",
    "path": "/users",
    "body": {
      "username": "anna"
    }
  },
  "response": {
    "status": 200,
    "body": {
      "uuid": "CA8BDwYICQ4JDAcOBAAKDg",
      "username": "anna"
    },
    "matchingRules": {
      "$.body.uuid": {
        "match": "type"
      }
    }
  },
  "providerState": "username anna is not already taken"
}

The Provider does not have to do anything special to take this change into account, so long as the Pact-verification implementation supports Matchers (the JUnit one does).

Pierre Vincent

Read more posts by this author.

France

Subscribe to Newsweaver Technology Blog

Get the latest posts delivered right to your inbox.

or subscribe via RSS with Feedly!