Frontend tests that pass on a local machine but fail after a release are often blamed on flaky selectors, timing issues, or “the test environment.” Sometimes that is true. But if the failures appear only after a CDN rollout, an asset hash change, or a service worker update, the root cause is often in the delivery layer, not in the UI logic itself.

That distinction matters. A test that breaks because the app bundle changed is not the same problem as a test that breaks because a button selector is unstable. In the first case, the app may be serving mixed versions of HTML, JavaScript, CSS, or cached assets. The test is exposing a deployment consistency issue, a cache invalidation issue, or a client-side update problem. Those failures can affect real users too, which is why they deserve the same rigor as software testing, test automation, and release validation practices that support reliable continuous integration.

Why delivery-layer regressions are so deceptive

Delivery-layer bugs are hard to diagnose because the application can look correct in one browser session and broken in another. A local run usually starts from a clean state, with a fresh build, no CDN edge cache, no stale service worker, and no browser history. The CI environment often behaves similarly, especially if each run uses a new container or ephemeral profile. Real users, and sometimes your test runners, do not.

A frontend release can introduce inconsistency in several places:

  • HTML served by the origin may reference new asset hashes.
  • A CDN may still cache old HTML or old chunks at the edge.
  • A browser may have an old service worker controlling navigation.
  • The app may preload or lazy-load chunks from the previous build.
  • API responses may be versioned differently than the UI bundle.

If a test fails only after deployment, ask whether the test is seeing a UI bug or a version-mismatch bug. The fix is often in caching, rollout sequencing, or service worker strategy, not in the test itself.

Start by proving whether the failure is version drift

Before changing the test, confirm that the browser is loading a consistent release. The simplest question is, “Does the page shell, JavaScript bundle, and CSS all come from the same build?”

You can inspect this in multiple ways:

1. Compare build identifiers end to end

Many teams embed a build ID or commit SHA into the HTML, response headers, or a global variable. If the page shell says build A, but the main bundle logs build B, you have a delivery mismatch.

A small runtime check can help:

<script>
  window.__BUILD_ID__ = '2026-07-13T10:22:00Z-abc123';
</script>

Then log it in the test:

import { test, expect } from '@playwright/test';

test('home page uses one build', async ({ page }) => {
  await page.goto('https://example.com');
  const buildId = await page.evaluate(() => (window as any).__BUILD_ID__);
  expect(buildId).toContain('abc123');
});

If the value changes between page loads or differs from the asset URLs, you have a reproducibility problem, not a locator problem.

2. Capture network requests during failure

When a test fails after rollout, inspect the request chain for the HTML document and static assets. In Playwright, this is often enough to show mixed versions:

page.on('response', async (response) => {
  if (response.request().resourceType() === 'script') {
    console.log(response.status(), response.url());
  }
});

Look for patterns such as:

  • HTML fetched from the latest deployment
  • JavaScript chunk fetched from an old CDN edge node
  • CSS loaded from a different cache path or older hash
  • 404s for chunks that no longer exist

3. Check whether the failure disappears in a fully uncached session

Open the same URL in a private window, disable browser cache, or use a new browser context in automation. If the test passes only in a clean session, caching or service worker state is likely involved.

Asset hash changes are supposed to help, until they do not

Hash-based filenames solve a real problem. They make it safe to cache static assets aggressively because a changed file gets a new URL. But the system only works if every layer updates in sync.

Common breakpoints include:

Stale HTML referencing removed chunks

If your HTML is cached too long, it can reference a chunk name from an older build. The browser then requests /assets/app.OLDHASH.js, which no longer exists after deployment.

This usually appears as:

  • blank page
  • ChunkLoadError
  • missing route content
  • hydration failure
  • test timeouts while waiting for the app shell to render

The test may fail at waitForSelector, but the real issue is a stale document or a stale preload link.

Mixed asset versions during rollout

If the CDN caches HTML and JS independently, a user may get new HTML but old JS, or the opposite. This is common when invalidation is partial, delayed, or scoped incorrectly.

A practical example is a UI test that clicks a menu item after deployment. The menu exists, but its event handler is missing because the loaded bundle is from an older release. The test reports a click failure, but the underlying issue is a version mismatch.

Build-time assumptions that break after deployment

Some apps inline a manifest or preload list that depends on exact asset names. If the HTML and asset manifest diverge, the app may boot with missing imports. Tests that wait for a visible component will simply time out.

What to check

  • Does the generated HTML reference hashed URLs that still exist on the origin?
  • Does the CDN invalidate the HTML and manifest together?
  • Are immutable assets cached with Cache-Control: public, max-age=31536000, immutable while HTML is short-lived?
  • Are service worker precache lists updated in the same release as the app shell?

Service worker updates add a second cache layer

Service workers are one of the most common reasons frontend tests fail after deployment. A service worker can keep serving old app assets long after the server has moved on.

A typical failure path looks like this:

  1. User or test runner loads version 1.
  2. Service worker caches app shell and chunks.
  3. Version 2 is deployed with new hashes.
  4. The page reloads, but the old service worker still controls the page.
  5. The app requests a chunk that no longer matches the cached manifest.
  6. The test fails on navigation, hydration, or async loading.

This is especially painful in CI if the same browser profile is reused across multiple tests or retries.

Debugging service worker state

In Playwright, you can inspect service workers in the browser context:

typescript

const sws = await page.context().serviceWorkers();
console.log(sws.map(sw => sw.url()));

If the app is controlled by an unexpected worker, or if the worker from a previous release is still active, you need to test update behavior explicitly.

Also check browser storage. A stale Cache Storage entry can survive until it is manually cleared.

Typical service worker mistakes

  • Precaching files without a reliable update strategy
  • Not calling skipWaiting() when appropriate
  • Not using clients.claim() when the app expects immediate takeover
  • Serving outdated cached responses for HTML navigation requests
  • Failing to version cache names per deployment

A realistic mitigation pattern

For applications that need immediate update behavior, you may combine versioned caches with explicit reload handling:

self.addEventListener('install', (event) => {
  self.skipWaiting();
});

self.addEventListener(‘activate’, (event) => { event.waitUntil(self.clients.claim()); });

This is not always the right product choice, but it makes deployment behavior less ambiguous for tests and for users.

CDN rollout issues are often consistency issues, not “CDN bugs”

When frontend tests fail only after a CDN rollout, the CDN is usually doing what it was configured to do. The problem is often configuration drift between origin, edge, and browser cache semantics.

Common rollout failure modes include:

Partial purge

The HTML is purged, but JS and CSS are not. Or the opposite. This can leave users with a document that points to missing assets or old chunks.

Wrong cache key

If query parameters, headers, or cookies are excluded from the cache key incorrectly, the CDN may serve a response built for another session, locale, or app version.

Inconsistent TTLs

If HTML stays cached too long at the edge, the app will continue to advertise old hash names. If assets are not cached long enough, you lose the benefit of immutable filenames and increase rollout pressure.

Origin and edge not aligned on headers

Different Cache-Control, ETag, or Vary headers can cause the browser and CDN to disagree about freshness.

Practical debugging checks

Use curl -I or browser devtools to compare headers from origin and edge:

curl -I https://example.com/
curl -I https://cdn.example.com/assets/app.abc123.js

Look at:

  • cache-control
  • age
  • etag
  • last-modified
  • via
  • x-cache
  • cf-cache-status or equivalent provider headers

You want to know whether the test fetched a fresh document, a cached document, or a response that appears fresh but contains old references.

How to separate test flakiness from deployment bugs

A useful rule is this, if the same test passes against a newly built local preview but fails against the deployed environment, suspect the delivery pipeline first.

Here is a simple triage sequence.

Step 1: Re-run against a known clean browser session

If the test passes with a fresh profile, the issue may be stale client state.

Step 2: Compare deployed response headers and asset URLs

If the URLs in the HTML do not match the release artifacts, you have a deploy mismatch.

Step 3: Clear browser storage and service worker state

If that makes the problem disappear, your release process is not handling client cache lifecycle correctly.

Step 4: Replay the build in a staging environment with production-like caching

Many teams test the app on localhost and miss edge behavior entirely. Staging should mimic CDN TTLs, immutable assets, and service worker scope as closely as possible.

Step 5: Verify the failing interaction with network throttling disabled and enabled

Some bugs only surface when an old chunk arrives late enough for the app to start booting against a mixed state.

Reproduce the bug with an explicit “old state, new deploy” scenario

The fastest path to a trustworthy fix is to recreate the mismatch on purpose. For example, use one browser context to load version 1, then deploy version 2, then reuse the same context and reload.

In Playwright, a lightweight reproduction might look like this:

import { test, expect } from '@playwright/test';

test('reload after deploy still renders', async ({ browser }) => {
  const context = await browser.newContext();
  const page = await context.newPage();

await page.goto(‘https://staging.example.com’); await page.waitForLoadState(‘networkidle’);

// Simulate a later deployment outside the test. await page.reload();

await expect(page.getByRole(‘heading’, { name: /dashboard/i })).toBeVisible(); });

That test is not meant to prove your deployment is working by itself. It is meant to surface assumptions about cache coherence, navigation timing, and update ordering.

Make your tests more diagnostic, not just more tolerant

It is tempting to add longer waits when tests fail after deployment. That can hide the symptom without fixing the cause. A better move is to make the failure more informative.

Use targeted assertions

Instead of waiting for a generic selector, verify the page state and the loaded version:

typescript

await expect(page.locator('meta[name="build-id"]')).toHaveAttribute('content', /abc123/);
await expect(page.locator('[data-testid="app-shell"]')).toBeVisible();

Capture console errors and network failures

Many asset mismatch bugs show up as console messages before the visible failure:

page.on('console', msg => {
  if (msg.type() === 'error') console.log(msg.text());
});

page.on(‘requestfailed’, req => { console.log(‘failed’, req.url(), req.failure()?.errorText); });

Assert on chunk load behavior

If your app has a known error page or reload fallback for missing chunks, test that behavior intentionally. A good deployment pipeline should fail safely when the browser has stale state.

Fixes that usually help, and what tradeoffs they create

1. Version everything consistently

Serve HTML, manifest, JS, CSS, and precache data from the same release version. This reduces mixed-state failures but requires stronger coordination in the pipeline.

2. Short-cache HTML, long-cache immutable assets

This is the standard model for hashed assets. HTML should be revalidated often, while content-addressed assets can be cached for a long time.

Tradeoff: if your rollout process does not invalidate HTML correctly, users may still see stale references.

3. Tie service worker updates to app versioning

Version the cache namespace and update strategy per release. This makes client state easier to reason about, but can increase update churn.

4. Add a deployment smoke test that checks version coherence

A smoke test should verify that a page load, asset fetch, and app boot all refer to the same build. This is often more valuable than a dozen brittle UI steps.

5. Provide a hard refresh or cache reset path for support and QA

If the app is difficult to recover from a stale worker or broken cache, make the reset path obvious. This reduces time spent diagnosing “random” test failures that are actually state persistence issues.

A practical CI/CD debugging checklist

When a frontend test fails after release, use this sequence before touching the test locator or wait strategy:

  • Confirm the failing run used the intended deployment SHA.
  • Inspect the HTML document and compare its asset URLs with the artifact manifest.
  • Check CDN headers for HTML and static assets.
  • Verify there are no 404s for chunk files.
  • Clear browser cache and service worker state, then rerun.
  • Compare behavior in a fresh browser context versus a reused one.
  • Review console errors for ChunkLoadError, hydration mismatch, or module resolution errors.
  • If the issue is isolated to one region, check CDN edge propagation and invalidation timing.

If the same browser session can fail after a deploy and recover after clearing storage, the test is often correct, but the release system is not deterministic enough.

What good release engineering looks like for frontend tests

The strongest teams treat frontend test failures after rollout as signals about release quality. They build observability around versioning, caching, and client update flow, not just around test pass rates.

That means:

  • build IDs are visible in the app and in logs
  • asset manifests are generated and validated in CI
  • CDN invalidation rules are documented and tested
  • service worker update behavior is exercised in staging
  • smoke tests run against the same cache assumptions as production
  • test environments avoid accidental reuse of stale browser state

If you already have a stable selector strategy and the test still fails only after deployment, the issue is probably not the selector. It is usually some combination of cache invalidation, asset hash changes, rollout sequencing, and client update state.

A final way to think about the problem

A frontend release is not complete when the backend says deploy succeeded. It is complete when a browser, arriving cold or warm, sees a coherent version of the application and can navigate without mixing old and new state.

That is why “frontend tests fail after CDN rollout” is such a useful diagnostic phrase. It points you toward the right layer. Instead of spending hours on waits and locators, check the build identity, the cache headers, the CDN invalidation path, and the service worker lifecycle. When those are correct, the UI tests often become stable without any special treatment.

If they are not correct, the tests are doing you a favor by failing early.