Content Pruning at Scale: Deciding What to Delete

On a site with hundreds of pages the per-page decision does not scale. The data to pull, the four buckets, the traps that destroy value, and why pruning reallocates attention rather than creating demand.

Deciding what to do with one page is a judgement. Deciding what to do with six hundred is a process, and most sites that attempt it either lose their nerve halfway or delete something that was quietly earning links.

The per-page framework is in content refresh. This is the version for when the list is too long to read.

When pruning is actually the answer

Four situations, and if none applies, you may not need to prune at all.

Crawl is being spent on pages that do nothing. On a site where important pages are not being indexed while hundreds of thin ones are being crawled, removing the thin ones reallocates attention. This is a real effect and it is the strongest case. The mechanism is in crawl budget explained.

Several pages compete for the same query. Consolidating them produces one stronger page. The diagnosis is in finding and fixing keyword cannibalisation.

Content is wrong. Stale prices, superseded rules, discontinued products, advice you no longer stand behind. Wrong pages damage trust in right ones.

The site has become unnavigable. A reader or a system trying to understand what you cover cannot, because the signal is buried in volume.

What pruning does not do is create demand. Removing three hundred pages does not make the remaining ones rank for things nobody searches. It redistributes the attention you already have. Expect that and you will judge the result correctly.

Pull the data first

You cannot decide from titles. For every URL on the site, assemble one row with the following.

  • Impressions and clicks over the last twelve months, from Search Console. Twelve months, because seasonal pages look dead for most of the year.
  • Index status. Whether it is indexed at all, which tells you whether the page is currently doing anything.
  • External links. Any page with inbound links from other sites is in a protected category regardless of its traffic.
  • Internal links in. How many of your own pages point at it.
  • Last modified date.
  • Word count, as a weak signal, not a decision.
  • Conversions or assisted conversions, if you have that connected. A page with no traffic that appears in the path of closed deals is not a candidate.

Pulling Search Console data for every URL is tedious by hand and straightforward by script, and the method is in exporting all your Search Console data with Python. Index status at scale is covered in bulk index checks.

Sort into four buckets

With the data in front of you, every page goes into one of four groups. Work through them in this order, because each one removes pages from consideration for the next.

Protect. Anything with external links, meaningful impressions, conversions in its path, or a clear role in navigation. These are not candidates, whatever their word count. Set them aside first so you cannot accidentally delete them later.

Improve. Pages with impressions but few clicks, or pages that are the only coverage of a topic you care about. They are earning attention and wasting it, which is an editing problem rather than a deletion one.

Merge. Groups of pages on the same subject. Pick the strongest URL in each group, by links and impressions, fold the useful content from the others into it, and redirect the rest to it. This is where most of the value in a pruning project usually sits.

Remove. What is left: no impressions, no links, no role, and often no reason anybody can remember. These go.

Expect the remove bucket to be smaller than you assumed when you started, and the merge bucket larger.

The traps

Each of these has destroyed real value on sites we have seen.

Deleting a page with external links. The links die with it. Redirect to the closest relevant page instead, or keep the page.

Judging by word count. A two-hundred-word page that answers one question precisely can be the most valuable page on the site. Length is not quality in either direction.

Judging by age. Old is not stale. A five-year-old page that is still accurate and still earning impressions is an asset.

Deleting the only page on a subject. If a topic matters to your business, zero coverage is worse than thin coverage. Improve it instead.

Removing pages that rank for long-tail queries nobody tracks. This is why you pull twelve months of impressions for every URL rather than relying on the ranking tool that watches forty keywords.

Redirecting everything to the home page. Treated as a soft 404, and the signals are lost anyway. Redirect to a genuinely related page or let it go honestly.

Execution, in batches

Do not remove three hundred pages on a Tuesday. Work in reviewed batches, watch, and continue.

  1. Start with the merge bucket, because consolidation produces visible improvement fastest and nothing is destroyed.
  2. For each merge, move the useful content, update internal links to point at the surviving URL, then redirect the others.
  3. For removals, decide per page between a redirect to a related page and a plain removal returning the right status. A page that genuinely has no successor should return a 410 or 404, not a redirect to something unrelated. The status question is covered in our field guide to HTTP status codes and the detection of soft 404s in 404s and soft 404s at scale.
  4. Remove every pruned URL from the sitemap in the same release.
  5. Find and update every internal link pointing at a removed page, so the site does not accumulate links to redirects. Redirect management at scale covers keeping that clean.
  6. Do a batch, wait three to four weeks, look at what happened, then do the next.

Keep a record of every URL and what was done to it. Six months later somebody will ask why a page is gone, and the answer should be in a spreadsheet rather than a memory.

What to expect afterwards

Be precise about this before you start, so nobody misreads the result.

Total impressions will usually fall slightly at first, because you removed pages that had a few. Impressions on the surviving pages should rise over the following weeks as attention redistributes. Crawl activity should shift towards the pages you kept, which is visible in log files if you have them, and the method for reading those is in log file analysis at scale.

Clicks and conversions should hold or improve, because you protected the pages that produced them.

If total clicks fall meaningfully, something in the protect bucket was misclassified, and the record you kept tells you which batch to look at.

Timescale is weeks to a few months. A site with a severe crawl problem shows change faster than a healthy one, because there was more waste to reallocate.

Rebuild the structure while you are there

Pruning is the moment to fix how the remaining pages connect, because you are already touching every internal link.

Consolidated topics should link to each other deliberately rather than incidentally. The approach we used on this site is in hub-and-spoke internal linking, and finding the links that should exist but do not is a solved problem with the script in internal link suggestions with embeddings.

What is a waste of effort

Pruning a small site. Under a hundred pages, read them and decide individually.

Pruning without the data pull. You will delete something that was working and keep something that was not.

Deleting pages to raise an average quality score in a tool. The tool’s score is not what ranks.

Treating pruning as the whole strategy. It frees capacity. What you do with the capacity is the strategy.

Doing it once and never again. Sites regrow. Annual review is proportionate.

A sensible order of work

  1. Confirm you have one of the four reasons to prune. If not, stop.
  2. Pull twelve months of impressions, clicks, index status, links and conversions for every URL.
  3. Fill the protect bucket first and set it aside.
  4. Sort the rest into improve, merge and remove.
  5. Execute merges first, in batches, with redirects and internal link updates.
  6. Execute removals with the correct status codes, and remove them from the sitemap.
  7. Record every action.
  8. Wait, measure the survivors, and continue.
  9. Fix the internal link structure of what remains.

If you want the data pulled and the buckets built for your site before anything is touched, get in touch.