How to Catch SEO Regressions Before Deployment

Automated checks that stop SEO mistakes before release: a tested smoke-test script for key URLs, Lighthouse CI budgets, crawl comparisons and redirect tests.

Ashruba Pankhade Published 5 min read

To catch SEO regressions before they reach production, add automated checks to your deployment pipeline: a smoke test that requests your key URLs on staging and fails the build if a page returns the wrong status, carries noindex, loses its canonical, title or H1, or has broken JSON-LD; a robots.txt check; and Lighthouse CI with performance budgets. These take an afternoon to set up and prevent the most expensive SEO mistakes, which are almost always accidental.

The regressions that cost the most

  • A staging noindex or “Disallow: /” reaching production
  • Canonical tags pointing to the staging domain or to the wrong page
  • Titles or H1s disappearing after a template change
  • Content moving into a client component and no longer rendering on the server
  • Structured data breaking because of a quote or character in the data
  • Redirects removed during a refactor
  • A large JavaScript bundle added, hurting Core Web Vitals

Check 1: an SEO smoke test

This Python script requests a list of key paths on a given host and checks status, noindex (meta tag and header), canonical host, title, exactly one H1, valid JSON-LD, and whether robots.txt blocks everything. It exits with an error if anything fails, so a CI job stops the deployment. It needs only Python 3.

#!/usr/bin/env python3
"""SEO smoke test for key URLs. Fails (exit 1) if a page regresses.
Usage: python3 seo_smoke_test.py urls.txt https://staging.example.com"""
import json, re, sys, urllib.request, urllib.error

def fetch(url):
    req = urllib.request.Request(url, headers={"User-Agent": "seo-smoke-test/1.0"})
    try:
        with urllib.request.urlopen(req, timeout=20) as r:
            return r.status, dict(r.headers), r.read().decode("utf-8", "replace")
    except urllib.error.HTTPError as e:
        return e.code, dict(e.headers or {}), ""

def check(base, path):
    url = base.rstrip("/") + path
    status, headers, html = fetch(url)
    problems = []
    if status != 200:
        problems.append(f"status {status}")
    robots_meta = re.search(r'<meta[^>]+name=["\']robots["\'][^>]+content=["\']([^"\']+)', html, re.I)
    if "noindex" in ((robots_meta.group(1) if robots_meta else "") + headers.get("X-Robots-Tag", "")).lower():
        problems.append("noindex")
    canon = re.search(r'<link[^>]+rel=["\']canonical["\'][^>]+href=["\']([^"\']+)', html, re.I)
    if not canon:
        problems.append("no canonical")
    elif not canon.group(1).startswith(base.rstrip("/")):
        problems.append(f"canonical on other host: {canon.group(1)}")
    title = re.search(r"<title[^>]*>(.*?)</title>", html, re.I | re.S)
    if not title or not title.group(1).strip():
        problems.append("empty title")
    if len(re.findall(r"<h1[\s>]", html, re.I)) != 1:
        problems.append("not exactly one h1")
    for block in re.findall(r'<script[^>]+application/ld\+json[^>]*>(.*?)</script>', html, re.I | re.S):
        try:
            json.loads(block)
        except ValueError:
            problems.append("invalid JSON-LD")
    return url, problems

def main():
    paths, base = [l.strip() for l in open(sys.argv[1]) if l.strip()], sys.argv[2]
    failed = 0
    for p in paths:
        url, problems = check(base, p)
        print(("FAIL " if problems else "ok   ") + url + ("  -> " + "; ".join(problems) if problems else ""))
        failed += bool(problems)
    robots = fetch(base.rstrip("/") + "/robots.txt")[2]
    if re.search(r"^\s*Disallow:\s*/\s*$", robots, re.M | re.I):
        print("FAIL robots.txt contains 'Disallow: /'"); failed += 1
    sys.exit(1 if failed else 0)

if __name__ == "__main__":
    main()

List one path per line in urls.txt: the home page, one of each template type and your highest-traffic pages. On staging, the canonical check compares against the staging host; run it again against production right after release.

It found a real problem on our own site

We ran the script against a local copy of this site while writing this article. It reported the blog listing page as “FAIL, no canonical”. WordPress adds canonical tags to posts and pages but not to blog listing and archive pages, and our theme did not add them either. We fixed the theme so listing, category and paginated pages carry correct canonicals, and the script now passes on every page. It is exactly the kind of gap nobody notices by looking at a page in a browser.

Check 2: Lighthouse CI

Lighthouse CI runs Lighthouse on your build and fails it if scores or metrics fall below your budgets:

// lighthouserc.json
{
  "ci": {
    "collect": { "url": ["http://localhost:3000/", "http://localhost:3000/services/seo/"], "numberOfRuns": 3 },
    "assert": {
      "assertions": {
        "categories:seo": ["error", { "minScore": 0.9 }],
        "categories:performance": ["warn", { "minScore": 0.8 }],
        "largest-contentful-paint": ["warn", { "maxNumericValue": 2500 }]
      }
    }
  }
}

Run it with npx @lhci/cli autorun in your pipeline. Lab scores differ from real-user data, so treat them as a guard against big regressions rather than a target.

Check 3: compare crawls before and after

For larger releases, crawl staging and production with the same settings and compare the exports: pages that disappeared, titles that changed, status codes that changed, internal links that broke. Any desktop SEO crawler that exports CSV works; a spreadsheet comparison is enough.

Check 4: redirects

Keep a file of important old URLs and their expected targets, and test that each returns a 301 to the right place on every deployment. Redirects are easy to lose in a refactor and expensive to lose in production.

Putting it in the pipeline

  1. Build and deploy to staging.
  2. Run the smoke test and Lighthouse CI against staging.
  3. Block the release if either fails.
  4. After release, run the smoke test against production.

Frequently asked questions

How many URLs should the smoke test cover?

Start with 10 to 30: one of each template and the pages that bring the most traffic and enquiries. The goal is speed and coverage of templates, not every page.

Should staging be crawlable by the script if it is password-protected?

Allow the CI runner through by IP or with a token header, while keeping staging closed to everyone else.

Does this help AI search as well?

Yes. The same regressions, especially noindex, robots.txt and client-side rendering, remove pages from AI search too. Pair it with our AI crawler access checker.

Sources

  1. Lighthouse CI documentation, github.com/GoogleChrome/lighthouse-ci
  2. Google Search Central, “Block Search indexing with noindex” and “How to specify a canonical URL”

Related reading