Performing Regression Testing for APIs with Playwright

Blog Image-10.svg

In modern API development, regression testing is crucial to ensure that new changes do not disrupt existing functionality. While Playwright is primarily recognized for front-end testing, it also offers robust capabilities for regression testing of APIs.

Introduction

Following my previous blog post about regression tests using Playwright, we will explore how to set up and use Playwright for API regression testing, key features that support regression workflows, and other Java-compatible alternatives suited for API regression testing. On this occasion, I would like to demonstrate how easy it is to use Playwright for creating regression tests for APIs.

Setting Up Playwright for Java API Regression Testing

The first step is to integrate Playwright with our Java project. This allows us to automate requests to our API, validate responses, and set up tests that ensure API stability with each new change.

Installation:

Add the Playwright dependency:

Maven:

<dependency>
    <groupId>com.microsoft.playwright</groupId>
    <artifactId>playwright</artifactId>
    <version>1.46.0</version>
</dependency>
Gradle:

groovy

dependencies {

implementation 'com.microsoft.playwright1.46.0'

}

Test Setup:

Allowing tests to adapt to different environments (e.g., dev, staging, production) I suggest configuring a base URL and other necessary variables as environment variables. This is essential for regression testing as we’ll often want to compare results across environments.

Creating Regression Tests:

I wrote two regression tests by sending an API request to a Weather API:

First, without a key, verify that the response contains an error message and not Ok status;

Then, with a key (token), verifying that the response has success status.

@Test
void should_returnError_when_noTokenProvided() {
//given
request = playwright.request().newContext(new APIRequest.NewContextOptions()
.setBaseURL(BASE_URL));
//when
var result = request.get("current.json");
//then
PlaywrightAssertions.assertThat(result).not().isOK();
assertThat(getApiError(result))
.isNotNull()
.isEqualTo(createNewAPIError());
}

@Test

void should_returnWeatherForecast_when_tokenIsValid() {

//given

request = playwright.request().newContext(new APIRequest.NewContextOptions()

.setBaseURL(BASE_URL));

//when

var result = request.get("current.json", RequestOptions.create()

.setQueryParam("key", API_KEY)

.setQueryParam("q", "Madrid"));

//then

PlaywrightAssertions.assertThat(result).isOK();

}

Key Playwright Features for Regression Testing

Playwright provides several features specifically designed to simplify the creation and maintenance of regression test suites:

  1. Automated Validation for Consistent Output: Use assertions to validate response bodies, headers, and status codes. This ensures changes in the API structure, such as field names or response codes, are detected quickly, a core focus in regression testing.
  2. Environment-Specific Testing with Variables: Store base URLs, API keys, and other environment-specific values outside the code, allowing tests to adapt dynamically between environments.
  3. Request and Response Mocks: For regression tests targeting endpoints with external dependencies, mocking specific responses helps isolate your API. This approach ensures the tests focus on your API's functionality rather than external integrations.
  4. Parallel Test Execution: Playwright’s ability to run tests concurrently supports regression testing for larger suites, allowing tests to finish quickly even as your API grows. This feature is valuable when performing regression testing as part of CI/CD pipelines, ensuring feedback loops remain short.

Writing Regression Test Scenarios with Playwright

Here are some effective strategies to consider when setting up your Playwright regression tests:

  • Core Functionality Verification: Use CRUD tests (Create, Read, Update, Delete) on key endpoints to ensure primary API functions remain stable. These tests validate both the response structure and content.
  • Boundary and Edge Case Testing: Regression tests should cover edge cases, such as empty requests, invalid data, and large payloads, to ensure your API handles a range of inputs consistently across releases.
  • Schema Validation: Implement schema validation for JSON responses to verify that field names and data types match expected formats. Changes here can signal breaking API changes.
  • Authentication and Security Checks: If your API uses token-based authentication, verify that authorization remains intact, especially after updates that could impact permissions.

Emilio Fulgencio

Emilio is a passionate and self-driven software engineer with over 20 years of IT experience. He has worked on Greenfield projects, Microservices, Batch processing, AI integration, OpenAPI, API-first approaches, monitoring systems, and asynchronous messaging.

Emilio excels in designing solutions, conducting technical spikes, and developing proof-of-concept initiatives. He adheres to best practices like Clean Code, TDD, KISS, and SOLID principles to deliver high-quality solutions.

A strong advocate for collaboration and knowledge sharing, as he believes, “When we teach someone, we learn more than we teach.”

author title
Author
author
  • Emilio Fulgencio