End-to-End Testing Every Rust Exercise with Playwright

The Rust platform has 71 exercises and counting (I just added a new track of Unix exercises). They all share the same interface: load an editor, type code, validate it against a Rust backend. When I make any changes to the platform, how do I confirm nothing breaks? Enter end-to-end testing with Playwright.

The Problem

Manual testing doesn't scale. Every time I add an exercise, tweak the editor, or update the validation flow, I need confidence that all exercises still work. Not just that the page loads, but the full loop: login, navigate, type code, submit, see results.

Unit tests cover the Django app, and the Rust validator has its own test suite. But neither exercises the full path a student takes: loading an exercise and getting a real pass or fail back.

One Test Function, 71 Test Cases

Playwright with pytest covers this in under 50 lines. Here's the core test file:

import psycopg2
import pytest
from decouple import config

from .constants import DOMAIN

exercises = []
with psycopg2.connect(dsn=config("DATABASE_URL")) as conn:
    with conn.cursor() as cursor:
        cursor.execute(
            "SELECT slug, solution FROM bites_exercise WHERE public = true"
        )
        exercises = cursor.fetchall()


@pytest.mark.parametrize("exercise", exercises, ids=[ex[0] for ex in exercises])
def test_exercise(logged_in_page, exercise):
    slug, solution = exercise
    page = logged_in_page

    exercise_url = f"{DOMAIN}/{slug}"
    page.goto(exercise_url)
    page.wait_for_url(exercise_url)

    page.wait_for_selector(".CodeMirror", state="visible")
    page.wait_for_function(
        "document.querySelector('.CodeMirror')?.CodeMirror !== undefined"
    )

    page.evaluate(
        f"""document.querySelector('.CodeMirror').CodeMirror.setValue({repr(solution)})"""
    )
    page.click("#validate-button")

    page.wait_for_function(
        "document.querySelector('#feedback').innerText.includes('Congrats') || "
        "document.querySelector('#feedback').innerText.includes('Oops')",
        timeout=30000,
    )

    validate_result = page.text_content("#feedback")
    assert "Congrats, you passed this exercise" in validate_result

The database query at module load fetches every public exercise with its solution. @pytest.mark.parametrize turns that into 71 test cases. Each test navigates, injects the solution, validates, and asserts success.

Here is Playwright running the tests locally against the real Rust validator, one exercise after another:

Patterns That Made It Work

Session-scoped fixtures for speed

Launching a browser is expensive. Logging in is expensive. Do it once:

@pytest.fixture(scope="session")
def browser(e2e_user):
    with sync_playwright() as p:
        with p.chromium.launch(headless=HEADLESS) as browser:
            yield browser


@pytest.fixture(scope="session")
def logged_in_page(browser):
    page = browser.new_page()
    page.set_default_timeout(30_000)
    page.goto(f"{DOMAIN}/pbadmin/")
    page.fill('input[name="username"]', LOGIN)
    page.fill('input[name="password"]', PASSWORD)
    page.click('input[type="submit"]')
    yield page

All 71 exercises run against the same authenticated browser session, so the login cost is paid once.

Waiting for CodeMirror

One tricky thing with Playwright is timing. Sometimes elements are not yet ready when you hit the page. In this case you have to wait for the CodeMirror editor to be fully initialized before injecting code:

page.wait_for_selector(".CodeMirror", state="visible")
page.wait_for_function(
    "document.querySelector('.CodeMirror')?.CodeMirror !== undefined"
)

page.evaluate(
    f"""document.querySelector('.CodeMirror').CodeMirror.setValue({repr(solution)})"""
)

First we wait for the selector to be visible, then we wait for the JavaScript instance to be ready.

Avoiding Django's async context trap

Another issue I faced was creating a Django user inside a Playwright fixture triggered:

SynchronousOnlyOperation: You cannot call this from an async context

I worked around it by creating the test user in a separate fixture that runs before Playwright starts:

@pytest.fixture(scope="session")
def e2e_user(django_db_blocker):
    with django_db_blocker.unblock():
        return ensure_e2e_user()


@pytest.fixture(scope="session")
def browser(e2e_user):  # e2e_user runs first
    with sync_playwright() as p:
        ...

The django_db_blocker.unblock() context manager allows database access in session-scoped fixtures. Order matters: the user must exist before the browser fixture runs.

Running Locally vs CI

The E2E suite runs against a live database with the Rust validator running. That's deliberate: for this layer I want real integration, not mocked responses. (The mocked cases have their own home, more on that below.)

# Run all 71 exercises
uv run pytest tests/test_e2e.py -v

# Debug a specific exercise
HEADLESS=False uv run pytest tests/test_e2e.py -v -k "exercise-slug"

By default, the tests run headless, which means no browser window opens. This is faster and works well in CI. If you want to see what Playwright is doing, set HEADLESS=False to open a visible browser window.

This is essential for debugging why a particular exercise fails. Use the -k option to filter for a specific exercise by slug. And you can use --pdb to leave the browser window open when a test fails, so you can inspect the state.

For CI, I run unit tests only. E2E tests require the Rust backend and take longer. I run them locally before pushing major changes. This is a good example of separating unit and integration tests.

What about the unhappy paths?

A reader pointed out that these E2E tests only cover the happy path: type the correct solution, see "Congrats". Fair observation. What happens when a student submits wrong code, or the validator itself blows up?

Those cases live in the unit tests, where I mock out the validator API call:

This is the split that makes the whole thing manageable. The E2E suite proves the full loop works end to end against the real validator. The unit tests, with the validator mocked, assert the exact message a student sees in each error case. Mocking is the natural home for these: some failure modes, like the runner throwing an exception, are hard to trigger on demand against a live backend, and even the ones you could reproduce in a browser are faster and clearer to pin down with a mock. See How to Tell if Your Python Mock Is Actually Working for the gotchas there.

What this buys me

Add an exercise, it's tested automatically, no new test code. That's the whole point of parameterizing over the database instead of hand-writing cases. Every frontend change now runs through a regression suite before it reaches users.

I find Playwright more modern and ergonomic than Selenium; the one rough edge is element timing, which the wait_for_function calls above handle. If you're testing anything with dynamic content, parameterizing over your real data beats writing one test per case.