Regression Testing with Playwright

Blog Image-10.svg

Regression testing is essential for maintaining software quality and ensuring that new features or code changes don’t introduce failures or alter expected behavior. However, performing these tests manually can be extremely tedious and time-consuming.

Introduction

Regression testing is essential for maintaining software quality and ensuring that new features or code changes don’t introduce failures or alter expected behavior. However, performing these tests manually can be extremely tedious and time-consuming.

By automating regression testing, you can efficiently verify that modifications to the application have not impacted existing functionality, helping to preserve overall system stability.

For Java developers, Playwright is an excellent tool for automating regression tests. Originally developed by Microsoft, Playwright is an end-to-end testing framework that supports modern web applications across various browsers.

Additionally, Playwright is adept at handling APIs, making it a versatile tool not only for browser automation but also for API testing. Playwright provides a built-in API to intercept, modify, and mock network requests, making it ideal for API-driven tests or scenarios requiring manipulation of backend calls.

In this article, we'll explore how to integrate Playwright with a Java-based project and write some regression tests. We'll also explore some alternative tools and compare their strengths and weaknesses. In a subsequent article, we'll focus on using Playwright for APIs.

Why should you consider using Playwright?

Playwright has become popular for several reasons:

  1. Cross-browser support: It supports Chromium, Firefox, and WebKit.
  2. Native handling of modern web features: Playwright offers first-class support for features like network interception, file uploads, and shadow DOMs.
  3. Headless and headed execution: You can run tests in a headless mode or with a visible browser, which is useful for debugging.
  4. Easy parallelization: It supports running tests in parallel, speeding up regression tests significantly.
  5. For Java developers, Playwright offers a Java binding that allows you to write tests in your preferred language while enjoying the benefits of the Playwright engine.
  6. Cross-language: Besides the Java option there are Playwright API in TypeScript, JavaScript, Python, and .NET.
  7. Powerful Tooling:
    1. Codegen: This tool records your actions in a browser and generates codes that can be used into any language.
    2. Playwright inspector[^1]: Inspect page, generate selectors, step through the test execution, view click points, and explore execution logs.
    3. Trace Viewer[^1]: Captures all the information to investigate the test failure. Playwright trace contains test execution screencast, live DOM snapshots, action explorer, test source, and many more.

Getting Started with Playwright in Java

In the following steps, we will learn how to integrate Playwright for regression testing in your Java project. You don't have to use the same project as your application; you can create a new project to use Playwright independently of your main project.

Add Playwright dependency

If you’re using Maven, add the following dependency to your pom.xml:

com.microsoft.playwright
playwright
1.46.0
If you're using Gradle, add this dependency to your build.gradle file:
implementation 'com.microsoft.playwright1.46.0'

Set Up Playwright Test

Playwright's API is user-friendly and suitable for both beginners and experienced developers. In the example below, I have created a dedicated project specifically for managing regression tests. However, Playwright can also be integrated into an existing project or organized as a standalone module if preferred.

To keep things simple, I have included a configuration class that initializes and shares instances of the Playwright and Browser classes. These instances are reused across all tests to create new pages for interaction. Below is an example of a basic configuration and a simple test performed on the GitHub website.

public class BaseConfiguration {

protected static Playwright playwright;

protected static Browser browser;

@BeforeAll

static void beforeAll() {

createPlaywright();

createBrowser();

}

@AfterAll

static void afterAll() {

closeBrowser();

closePlaywright();

}

private static void closePlaywright() {

if (playwright != null) {

playwright.close();

}

}

private static void closeBrowser() {

if (browser != null) {

browser.close();

}

}

private static void createBrowser() {

browser = playwright.chromium().launch();

}

private static void createPlaywright() {

playwright = Playwright.create();

}

}

This is a simple test using Playwright:

@Test

void should_containsTitle_when_accessMainPage() {

//given

var url = "https://github.com";

var page = browser.newPage();

//when

page.navigate(url);

//then

PlaywrightAssertions.assertThat(page)

.hasTitle("GitHub: Let’s build from here · GitHub");

}

In this example, I am adopting a "baby steps" approach to ensure the test is easy to understand and simple to implement, focusing on one task at a time. This method is especially beneficial for developers who are new to Playwright or browser automation.

I am also using the Given-When-Then pattern, a well-established structure in Behavior-Driven Development (BDD). This pattern effectively delineates the different phases of a test scenario:

  • Given: Establishes the initial context or precondition.
  • When: Describes the action or event that occurs.
  • Then: Specifies the expected outcome or result.

This structure enhances readability, particularly in complex test cases, and is widely adopted across the industry within testing frameworks like Cucumber and SpecFlow.

Writing tests

To simplify the process of creating your own tests, Playwright provides a helpful tool called Inspector. This tool lets you visually track each step of your test execution and take advantage of code generation by recording your actions in the browser. In the next section, I will demonstrate how to make the browser visible and use the Inspector to generate functional tests.

Turn browser visible

In this code fragment, we will create a new page object to display the browser,`setHeadless(false)`, and allow interaction with the page.

var page = playwright.chromium()

.launch(new BrowserType.LaunchOptions()

.setHeadless(false)).newPage();

//when

page.navigate(GITHUB_URL);

page.pause();

When you run this test the browser window will be visible:

github_main_page.png

And the Inspector window will be visible as well.

inspector.png

The Inspector window will list all steps. To generate interactive code, click the Record button and start navigating through the page. You should see a result like this:

inspector-with-steps.png

It will quickly move to your code and allow you to create tests in a fast way. Please note that while this code may not follow the best structure, it provides a solid starting point for interacting with the UI.

The test could resemble the code snippet below:

//given

var page = browser.newPage();

page.navigate(GITHUB_URL);

//when - It could be clearer to create a method to find a project and receive the value as a parameter,

// but for test purposes, I will leave this code here.

page.getByLabel("Search or jump to…").click();

page.getByRole(AriaRole.COMBOBOX, new Page.GetByRoleOptions().setName("Search")).fill("playwright");

page.getByRole(AriaRole.COMBOBOX, new Page.GetByRoleOptions().setName("Search")).press("Enter");

page.getByRole(AriaRole.LINK, new Page.GetByRoleOptions().setName("microsoft/playwright-java")).click();

//then

PlaywrightAssertions.assertThat(page).hasURL("https://github.com/microsoft/playwright-java");

This code example illustrates how easy it is to create UI regression tests using the Playwright Inspector. There is ample opportunity to improve the code’s readability

Playwright Alternatives

Playwright is a powerful tool for regression testing, but it's important to explore alternatives to find the best fit for each use case.

Selenium

Selenium has been a popular choice for web testing for a long time. It supports multiple programming languages, including Java, and provides strong browser automation capabilities. However, compared to Playwright, it lacks some modern web automation features, such as network interception and native shadow DOM handling. Selenium can also be slower, especially when running parallel tests.

Pros:

  • Widely supported with a large community and ecosystem.
  • Works with multiple browsers.

Cons:

  • Slower test execution compared to Playwright.
  • Lacks built-in parallel execution.

Cypress

Cypress is a fast and user-friendly testing tool that primarily targets JavaScript applications. It is well-regarded for its fast execution and clear error messages. While it may not be as comprehensive as Playwright for multi-browser testing, Cypress offers an excellent user interface for debugging.

Pros:

  • Excellent for JavaScript-heavy applications.
  • Fast and reliable.

Cons:

  • JavaScript only, so not suitable for Java-based projects.
  • Limited support for cross-browser testing (no WebKit support).

TestCafe

TestCafe is another excellent tool for browser automation that operates without WebDriver. It is easy to set up and supports JavaScript. Like to Cypress, it has limited language support, primarily focusing on JavaScript environments

Pros:

  • Simple and fast.
  • No need for browser plugins or WebDriver.

Cons:

  • JavaScript only.
  • Lacks deep browser integration that Playwright offers.

Puppeteer

Puppeteer is a Node.js library that provides an API to control Chrome or Chromium over the DevTools Protocol. It is similar to Playwright but doesn’t support multiple browsers. Puppeteer is an excellent choice for testing Chrome-based applications.

Pros:

  • Fast and lightweight.
  • Ideal for Chromium-based tests.

Cons:

  • Chromium-only support.
  • Limited compared to Playwright’s browser coverage.

Emilio Fulgencio

Emilio is a passionate and self-driven software engineer with over 20 years of IT experience. He has worked on Greenfield projects, Microservices, Batch processing, AI integration, OpenAPI, API-first approaches, monitoring systems, and asynchronous messaging.

Emilio excels in designing solutions, conducting technical spikes, and developing proof-of-concept initiatives. He adheres to best practices like Clean Code, TDD, KISS, and SOLID principles to deliver high-quality solutions.

A strong advocate for collaboration and knowledge sharing, as he believes, “When we teach someone, we learn more than we teach.”

author title
Author
author
  • Emilio Fulgencio