What Is a Broken Link? The 2026 Fix Guide (38% Rot)

By Alex Bain 2026-08-27 21 min read

A broken link is a hyperlink that no longer reaches its intended destination, and the scale of the problem is bigger than most teams assume: 38% of the webpages that existed in 2013 were already gone by 2023, according to the Pew Research Center. When a link breaks, a browser or a search crawler asks a server for a page and receives an error such as a 404 Not Found or a 500 Internal Server Error instead of content. For a visitor, that is a dead end. For a search engine, it is wasted crawl budget and a weaker trust signal. For a B2B team, it is lost conversions, decayed backlinks, and in regulated sectors, compliance exposure. This guide shows you how to find, fix, and prevent them.

What is a broken link, exactly?

A broken link, also called a dead link, is any hyperlink whose target resource cannot be retrieved when someone or something requests it. The visible anchor still looks clickable, but the address it points to returns an error or a dead-end page instead of the intended content. The vendor Dr. Link Check defines the problem the same way: hyperlinks that no longer lead to their intended destination, usually because a page was moved, renamed, or deleted without a redirect.

Links break for a small number of repeatable reasons. The destination page was deleted. The URL was changed during a redesign or a content-management-system migration and no redirect was created. Someone mistyped the href in the markup. A domain expired or changed hands. The server that hosts the target went down or started blocking automated requests. A file such as a PDF or an image was removed from storage. An HTTPS migration left insecure or mixed-content references behind. Each of these produces the same symptom from the requester's point of view: the resource is not there.

It helps to separate the label from the cause. A link is a promise that a specific address will return a specific resource. A broken link is a promise the web can no longer keep. Rowan University's web team put it plainly in a March 2026 guidance post.

"A broken link is a digital dead end." Rowan University Web Services, March 2026, which advises teams to tag outdated content for removal or archival, or to convert a stale PDF into an HTML page, rather than leave a dead link in place.

How a link request actually works

Understanding the mechanics makes broken links easy to diagnose. When a link is followed, the client resolves the domain through DNS, opens a TCP and TLS connection, and sends an HTTP request for the path. The server answers with a status line that carries a three-digit code. Codes in the 200 range mean success, the 300 range means a redirect, the 400 range means the client asked for something that is missing or forbidden, and the 500 range means the server failed. A broken link is any request that ends in a 400 or 500 range response, a redirect chain that never resolves, a redirect loop, or a soft 404, which is a page that returns a 200 success code while telling the user the content does not exist. Every method in this guide is a way of surfacing those responses at scale.

Why broken links matter for SEO, revenue, and AI citations

Broken links are not a cosmetic issue. They touch crawl efficiency, rankings, revenue, and, in 2026, whether AI answer engines will cite you at all. Start with crawling. Search engines allocate a finite crawl budget to every site. When crawlers spend that budget hitting dead URLs and soft 404s, they index less of your valuable content. Google's own guidance is explicit that returning a proper 404 or 410 for missing pages improves the crawl coverage of your best content, while soft 404s waste crawl budget because Googlebot keeps revisiting them.

Then there is link equity. A backlink is only valuable if the page it points to resolves. When an inbound link lands on a 404 on your site, the authority that link carried is stranded. Ahrefs has estimated that the average website loses roughly 5 to 6 percent of its backlinks every year to link rot, redirects that expire, and deleted content. Every one of those is equity leaking out of your domain that a single 301 redirect could have reclaimed. This is why broken-link cleanup is a core part of any serious technical SEO program, not an afterthought.

Ahrefs analyzed the link profiles of more than 2 million domains and found that 66.5% of the links pointing to those sites since January 2013 are dead, rising to 74.5% once SEO-related losses such as deindexed pages and soft 404s are included.

Revenue is the most direct cost. A broken call-to-action, a dead pricing link, or a 404 on a product page sends a ready-to-buy visitor into a wall. Conversion work assumes the path exists. Finally, the 2026 dimension: AI answer engines such as Google AI Overviews, ChatGPT search, and Perplexity preferentially cite live, retrievable URLs. A source that returns a 404 gets dropped from the answer, so link integrity now feeds directly into answer engine optimization and whether your brand appears in AI-generated results. Broken links quietly erode all four at once.

The 2026 link rot data every SEO should know

Before you audit a single URL, it helps to internalize how common broken links really are, because the numbers change how you prioritize. Link rot is not an edge case that affects neglected sites. It is the default entropy of the web, and it accelerates with a page's age. The most rigorous public dataset comes from the Pew Research Center's May 2024 study, "When Online Content Disappears," which sampled pages that existed between 2013 and 2023. The second major dataset comes from Ahrefs, which analyzed the link profiles of more than 2 million domains. Read together, they show that a meaningful share of every link graph is already dead, and that news, government, and reference sites are not immune.

"A quarter of all webpages that existed at one point between 2013 and 2023 are no longer accessible, as of October 2023." Pew Research Center, "When Online Content Disappears," May 17, 2024.

The table below consolidates the headline figures. Treat these as base rates. If you have never audited your site, assume a similar fraction of your outbound and inbound links have already rotted.

The pattern to take away is that scale makes rot worse, not better. Ahrefs found that larger sites had proportionally more link rot than smaller ones, because more pages means more inbound references pointing at URLs that eventually change. If you run a large content library, a documentation set, or a decade of blog archives, your broken-link count is almost certainly in the hundreds or thousands, and most of it is invisible until you crawl.

HTTP status codes that reveal a broken link

Every broken-link audit is ultimately a hunt for the wrong status code. The status code is the server's one-line verdict on your request, standardized in RFC 9110, the HTTP Semantics specification published in June 2022, and documented in plain language by MDN Web Docs. Learning to read these codes is the single most useful skill for triaging links, because the code tells you not just that a link is broken but why, and therefore how to fix it. A 404 needs a different response than a 503, and a 301 chain needs a different response than a hard 410.

The table maps the codes you will see most often in a crawl, whether each one signals a broken link, and the first action to take. Codes in the 400 and 500 families are almost always genuine breaks. The 300 family is trickier: a single 301 is healthy, but chains and loops behave like breaks.

Hard 404 versus soft 404 versus 410

These three are the codes teams most often get wrong. A hard 404 is correct behavior for a page that does not exist: the server returns a 404 status and the crawler learns to stop indexing it. A 410 Gone is a stronger signal that says the resource was intentionally removed and will not come back, which can prompt faster de-indexing. A soft 404 is the dangerous one. The server returns a 200 success code, so tools think the page is fine, but the visible content says the page cannot be found. Google treats soft 404s as errors, wastes crawl budget on them, and may index the empty page. The BrokenLinkCheck.com team describes the standard technical symptom as an error response such as a 404 for a missing page or a 500 for a server error, which is exactly the signal a soft 404 hides. Always make sure removed pages return a real 404 or 410, not a 200 with a friendly message.

The seven types of broken links you will find

Not all broken links are equal, and a triage plan starts by sorting them by type, because each type has a different owner, a different fix, and a different business impact. An internal broken link is your responsibility and your fastest win. An inbound broken backlink is equity you can reclaim. A broken outbound link is a trust and user-experience problem. When you export a crawl, tag every broken URL with one of the categories below before you decide what to do. The sorting takes minutes and saves hours, because it stops you from treating a decorative footer link the same way you treat a dead pricing page.

Here are the seven types you will encounter in almost every audit, in rough order of how often they cause real damage:

• Internal broken links. Links between pages on your own domain that point to a moved or deleted URL. These are fully in your control and should be fixed first.
• Outbound broken links. Links from your pages to external sites that have since removed or moved the target. They hurt user trust and, at scale, editorial quality.
• Inbound broken backlinks. Links from other sites that point to a URL on your domain that no longer resolves. Each one is stranded link equity you can recover with a redirect.
• Broken image and resource links. Missing images, stylesheets, scripts, or fonts that return a 404. They break layout and rendering, not just navigation.
• Broken anchor and fragment links. Links to a specific section identifier on a page where that section no longer exists, so the jump silently fails.
• Redirect chains and loops. A link that passes through several hops before resolving, or that loops forever, wasting crawl budget and slowing users.
• Soft 404s and mixed content. Pages that return a 200 code with missing content, and insecure references left behind after an HTTPS migration.

Once every broken URL carries a type tag, you can route the work: engineering fixes redirects and server errors, content owners fix outbound links and anchors, and your SEO or link building team focuses on reclaiming inbound backlinks, which is where the measurable ranking upside lives.

Prerequisites: the exact tools and versions you need

You can run a competent broken-link audit with free tools, but a professional workflow combines a desktop crawler, the search console, a backlink index, and a scripting environment for automation. Below is the stack this guide uses, with the versions current as of August 2026. You do not need every item to start, but you will want the crawler and the search console at minimum. The Python tools matter later, when you automate monitoring so you never ship a broken link again.

Confirm you have the following before you begin:

• Screaming Frog SEO Spider 24.3, the current release as of mid-2026. The free edition crawls up to 500 URLs. The paid license is 199 GBP, about 279 USD or 259 EUR, per user per year, and removes the crawl cap and unlocks scheduling.
• Python 3.12 or 3.13, for the scripts in this guide. Both are supported and stable in 2026.
• The requests 2.32.x library and beautifulsoup4 4.12.x, installed with pip, for the synchronous checker.
• httpx 0.27.x, for the asynchronous monitor in the final project, which checks links far faster than a serial loop.
• Google Search Console, free, verified for your domain, for the Pages report and the historical 404 data that crawlers cannot see.
• Ahrefs or Semrush, a paid subscription, for finding inbound broken backlinks that point at dead URLs on your domain.
• curl 8.x, preinstalled on macOS and Linux and available on Windows, for one-off status-code checks from the command line.
• A spreadsheet, Google Sheets or Excel, for the triage list that turns a raw crawl export into an assignable task list.

If you want zero-install options for a quick first pass, two browser-based scanners are worth bookmarking: BrokenLinkCheck.com and the W3C Link Checker. They are slower and less configurable than a desktop crawler, but they need nothing but a URL, which makes them a fair way to prove the problem exists before you invest in the full stack.

How to find broken links, step by step

This is the core workflow. It moves from a full desktop crawl, to the search console, to targeted command-line checks, to the backlink index, and ends with a triage sheet you can hand to engineering and content owners. Follow the steps in order the first time. Once you know the tools, you will run steps 2 through 6 in parallel. Each step includes what you will see on screen or in the output, so you can confirm you are in the right place.

1. Define the scope. Decide whether you are auditing the whole domain, a subdomain, or a single high-value section such as /pricing or /product. Write the start URL down. For a first audit, crawl the whole site so you get a true broken-link count.
2. Run a full crawl in Screaming Frog. Open the SEO Spider, paste your start URL into the address bar at the top, and click Start. You will see the URL count climb in the bottom status bar and rows populate in the main grid in real time. Wait for the progress bar to reach 100%.
3. Filter the Response Codes tab for client errors. Click the Response Codes tab, then set the filter dropdown to Client Error (4xx). The grid now shows only URLs that returned 400 to 499. The Status Code column will read 404, 403, or 410, and the Status column will read Not Found or Forbidden.
4. Repeat for server errors. Switch the filter to Server Error (5xx) to surface 500, 502, 503, and 504 responses. These often point to application bugs or overloaded services rather than missing pages, so tag them for engineering.
5. Find the source pages with Inlinks. Click any broken URL, then open the Inlinks tab in the lower window. It lists every page that links to the broken URL, plus the exact anchor text and link position. This is the list of pages you actually have to edit.
6. Bulk export the evidence. Use Bulk Export, then Response Codes, then Client Error (4xx) Inlinks and Server Error (5xx) Inlinks. Screaming Frog writes a spreadsheet where every row is one broken link with its source page, destination, anchor text, and status code.
7. Check outbound external links. In the same crawl, set the tab filter to External and the response filter to Client Error (4xx). This surfaces dead links pointing off your site. Export these separately, because content owners fix them, not engineering.
8. Cross-check Google Search Console. Open the Pages report under Indexing. Look at the Not indexed reasons, especially Not found (404) and Soft 404. Screenshot description: a bar chart of indexed versus not-indexed pages, with a table of reasons below and an example URL list you can expand per reason. This catches URLs that crawlers reach through external links but your internal crawl never touches.
9. Spot-check with curl. For any URL you are unsure about, run a header request from the command line to see the true status code without a browser cache in the way. The output shows the status line first.
10. Find inbound broken backlinks in Ahrefs or Semrush. Open Site Explorer, enter your domain, and go to the Best by Links report filtered to 404 Not Found. This lists dead URLs on your domain that still have external links pointing at them, sorted by referring domains. These are your highest-value fixes.
11. Run a free scanner as a sanity check. Point Dr. Link Check or BrokenLinkCheck.com at a key section to confirm your crawler is not missing anything obvious. Two independent tools agreeing gives you confidence in the count.
12. Consolidate into a triage sheet. Merge every export into one spreadsheet with columns for source URL, broken URL, type, status code, referring domains, and fix action. Sort by business value: pages with traffic and backlinks first, decorative links last.

The output of step 9 looks like the header check below, which is the fastest way to confirm a single broken link in isolation:

$ curl -I -A 'LinkAudit/1.0 (+
)' 

HTTP/2 404
date: Thu, 27 Aug 2026 14:02:11 GMT
content-type: text/html; charset=UTF-8
server: nginx

# The 404 in the status line confirms the page is a broken link target.

Build a broken link checker in Python

Desktop crawlers are excellent for a scheduled audit, but you will often want a script you can run in a pipeline, point at a single page, or wire into a deployment check. Python makes this a short job. The two building blocks are a function that asks for a URL's status code and a crawler that extracts every link from a page and checks each one. Start with the single-URL check. It sends a lightweight HEAD request, follows redirects to the final destination, and classifies anything that returns 400 or higher, or that throws a network error, as broken.

import requests

def check(url):
    try:
        r = requests.head(url, allow_redirects=True, timeout=10,
                          headers={'User-Agent': 'LinkAudit/1.0'})
        if r.status_code >= 400:
            return 'BROKEN', r.status_code
        return 'OK', r.status_code
    except requests.RequestException as e:
        return 'ERROR', str(e)

print(check('
.com/pricing'))
# -> ('BROKEN', 404)

HEAD requests are polite and fast because they ask only for headers, not the page body. Some servers reject HEAD or answer it incorrectly, so a production script should fall back to a GET request when it sees a 405 Method Not Allowed. The final project later in this guide does exactly that.

To audit a whole page, extract its links with BeautifulSoup and run each through the same check. The crawler below reads one page, finds every anchor with an href, resolves relative URLs against the page address, and records anything that fails. It deduplicates with a set so it never checks the same URL twice.

import requests
from bs4 import BeautifulSoup
from urllib.parse import urljoin

START = '
.com'
seen, broken = set(), []

def links_on(page):
    html = requests.get(page, timeout=15,
                        headers={'User-Agent': 'LinkAudit/1.0'}).text
    soup = BeautifulSoup(html, 'html.parser')
    for a in soup.select('a[href]'):
        yield urljoin(page, a['href'])

for url in links_on(START):
    if url in seen:
        continue
    seen.add(url)
    try:
        code = requests.head(url, allow_redirects=True, timeout=10).status_code
    except requests.RequestException:
        code = 0
    if code == 0 or code >= 400:
        broken.append((url, code))
        print('BROKEN', code, url)

print(len(broken), 'broken links found')

This is enough to check a landing page or a blog post in seconds. To crawl an entire site you add a queue of internal URLs and a rule to stay on your own domain, which is the shape of the complete project at the end of this guide. Keep the timeout at 10 to 15 seconds and set a descriptive user-agent so server owners can identify your checker in their logs.

How to fix broken links the right way

Finding broken links is the easy half. Fixing them well means matching the response to the intent, because the wrong fix creates new problems. Redirecting a deleted page to your homepage, for example, looks tidy but Google often treats a redirect to an irrelevant page as a soft 404, so you gain nothing. Use the decision list below to route every broken link to the correct fix. Work top to bottom through your triage sheet, highest business value first, and record the action you took so you can verify it in the next crawl.

• The page moved or was renamed. Create a 301 permanent redirect from the old URL to the new one. This preserves link equity and sends users straight to the right content.
• The content is gone and nothing replaces it. Return a 410 Gone, or a 404, and remove the internal links that point to it. Do not redirect it to an unrelated page.
• There is a close equivalent. Redirect the dead URL with a 301 to the most relevant live page, not to the homepage, so the redirect is a genuine match.
• The link has a typo. Fix the href in the source markup. No redirect is needed because the correct URL already works.
• An outbound target moved. Update the link to the new address, or point it at an archived copy in the Wayback Machine if no live equivalent exists.
• An inbound backlink hits a dead URL. 301 that old URL to the best live equivalent to reclaim the external equity, then, if the link is valuable, ask the linking site to update it.

The redirects themselves live in your server config. The examples below show a single-URL 301 and a pattern-based 301 in Apache, plus the Nginx equivalents and a deliberate 410 for content you removed on purpose.

# Apache .htaccess: 301 a moved page to its new location
Redirect 301 /old-pricing /pricing

# Apache: pattern-based 301 for a renamed directory
RewriteEngine On
RewriteRule ^guides/(.*)$ /resources/$1 [R=301,L]

# Nginx: permanent redirect for a single URL
location = /old-pricing {
    return 301 /pricing;
}

# Nginx: return 410 Gone for content removed on purpose
location = /discontinued-product {
    return 410;
}

For outdated assets specifically, Rowan University's advice is worth repeating: if content is stale, the durable fix is often to delete it, archive it, or convert a hard-to-maintain PDF into an HTML page, rather than patch a link to a file that will rot again. A redirect is a fix. Removing the source of future rot is a cure.

Common pitfalls and how to fix them

Teams rarely fail to find broken links. They fail in how they fix them, and the same handful of mistakes shows up in audit after audit. Each pitfall below includes the symptom you will notice and the concrete correction. Reviewing this list before you start editing redirects will save you a second cleanup pass, because most of these errors are invisible until the next crawl or, worse, until rankings dip and you go looking for the cause.

• Pitfall: redirecting everything to the homepage. Symptom: dozens of old URLs all 301 to the root. Fix: redirect each dead URL to its closest live equivalent. Google frequently classifies a redirect to an unrelated page as a soft 404, so the mass redirect passes no value and can be dropped from the index.
• Pitfall: redirect chains. Symptom: a link passes through two, three, or more hops before it resolves. Fix: collapse every chain to a single hop by pointing the original URL directly at the final destination. Chains waste crawl budget and slow down users on mobile connections.
• Pitfall: ignoring trailing slashes and case. Symptom: /Page and /page, or /path and /path/, resolve differently and one of them 404s. Fix: pick one canonical form, enforce it with a redirect rule, and make internal links match it exactly.
• Pitfall: using 302 for permanent moves. Symptom: moved pages return a temporary 302. Fix: change permanent moves to 301 or 308 so search engines transfer signals to the new URL instead of holding them on the old one.
• Pitfall: crawling only raw HTML. Symptom: links injected by JavaScript never appear in the crawl, so client-side broken links slip through. Fix: enable JavaScript rendering in your crawler so it evaluates the rendered page, not just the initial markup.
• Pitfall: fixing internal links but ignoring inbound backlinks. Symptom: your site is clean internally but external sites still link to dead URLs. Fix: pull the 404 backlink report from Ahrefs or Semrush and 301 those URLs to reclaim the equity, which is often the single highest-ROI action in the whole audit.

The through-line is that a broken-link fix is only correct if it is relevant, permanent, and verified. After any batch of redirects, recrawl the affected URLs and confirm each one now returns a 200 in a single hop. A fix you did not verify is a hypothesis, not a fix.

Troubleshooting: eight problems and their fixes

Broken-link audits throw the same operational problems repeatedly, usually because the crawler behaves differently from a real browser, or because the server treats bots differently from humans. When a result looks wrong, do not assume the link is fine or broken until you have ruled out these causes. The table below lists the issues that account for most confusion during an audit, the likely reason behind each, and the fix. Keep it next to your triage sheet, because a single misdiagnosed 403 can send an engineer chasing a bug that does not exist.

Two of these deserve extra attention because they cause the most wasted effort. The 403-in-crawler-only problem is nearly always a bot-protection layer, so before you file a bug, retry the URL with a browser-like user-agent and confirm it still fails. The Search-Console-only 404s are frequently links from other sites or from your own historical URLs that no live page references, so weigh each one by referring domains before spending time on it. Fixing a 404 that has no traffic and no backlinks is motion, not progress.

Broken link building: turn dead links into backlinks

Broken links are usually a liability, but one tactic turns them into an asset. Broken link building is the practice of finding dead outbound links on other people's pages, creating or already owning a working replacement, and asking the site owner to swap the dead link for yours. It works because you are doing the site owner a favor: you found an error on their page and handed them the fix. That reciprocity, plus a genuinely relevant replacement, is why the tactic still earns links in 2026 when generic outreach is ignored. Given that Ahrefs found 66.5% of older links are already dead, the supply of opportunities is effectively unlimited.

The workflow is mechanical once you have the tools. Here is the sequence that consistently produces placements:

1. Pick a target topic and find resource pages. Search for roundup and resource pages in your niche, the kind that link out to many external sources, because they rot the fastest.
2. Crawl those pages for dead outbound links. Run each candidate through Screaming Frog in list mode, or a free checker, and filter for 4xx and 5xx external links.
3. Confirm the dead target's original topic. Use the Wayback Machine to see what the dead URL used to contain, so you know what the replacement needs to cover.
4. Match or create the replacement content. If you already have a page that fits, use it. If not, and the opportunity is large enough, publish content that covers the same ground better.
5. Find the right contact and send a specific email. Reference the exact broken link and the page it sits on, then offer your replacement as a fix.

A short, specific outreach note outperforms a template. The structure below works because it leads with the value you are providing, not the link you want.

Subject: broken link on your [topic] resource page

Hi [name],

I was reading your guide at [page URL] and noticed the link to
[dead URL] returns a 404. The original page looks gone.

We published a current version that covers the same ground here:
[your URL]. Feel free to use it as the replacement if it helps
your readers.

Either way, thanks for the resource.
[your name]

This is a repeatable, measurable channel, which is why it sits at the center of most structured link building programs. If you want it run at scale with vetted targets and managed outreach, that is exactly the kind of campaign our growth and demand team builds. Done well, broken link building converts the web's natural decay into referring domains for your site.

Advanced tips for auditing at scale

Once you can find and fix broken links, the goal shifts from cleanup to prevention, and from one-off audits to continuous monitoring. Large sites cannot afford a manual crawl every week, and the highest-value URLs, the ones with traffic and backlinks, deserve tighter watch than the long tail. The tips below are the practices that separate a one-time cleanup from a durable program that keeps your link graph healthy as the site grows and content ages.

• Monitor continuously, not occasionally. Schedule the async monitor from the next section on a cron job and post results to Slack, so a new broken link is caught within a day of shipping, not at the next quarterly audit.
• Log 404s from real traffic. Fire a GA4 event on your 404 page or parse server logs, so you see the dead URLs that actual users and Googlebot hit, ranked by frequency, which is a better priority list than a raw crawl.
• Build a useful 404 page. Include site search, links to top pages, and a clear message, so the inevitable broken link recovers the visitor instead of losing them.
• Use the Wayback Machine to recover targets. When an outbound link dies, its archived copy often reveals the original content, so you can find a current equivalent instead of just deleting the link.
• Prioritize by equity and traffic, not raw count. A single 404 with fifty referring domains matters more than five hundred dead links in a footer. Sort every audit by business value.
• Analyze server log files. Log analysis shows exactly which URLs Googlebot requests and which return 404, which is the ground truth of what search engines experience on your site.
• Archive your own outbound links at publish time. Submit important external references to the Wayback Machine when you publish, so if they rot later you already have a durable copy to point to.
• Segment large crawls. Crawl by directory or by sitemap for very large sites, so a single run does not overload the origin server or exhaust your own memory.

The common thread is that prevention is cheaper than remediation. A monitor that alerts you the day a link breaks costs a few minutes to set up and saves the compounding damage of a dead link that sits live for months, quietly bleeding crawl budget, equity, and conversions. This is the same principle that underpins any mature technical SEO practice: catch the regression at the source.

Broken links as a compliance and regulatory risk

For most sites a broken link is a marketing problem. In regulated industries it can become a filing-quality or corrective-action problem, because disclosure rules assume that the documents and links they reference stay reachable for years. Link integrity is not only about rankings when your links sit inside legal filings, regulatory disclosures, or price-transparency files that an agency can audit. Three examples show how far the risk extends beyond SEO.

First, securities filings. In April 2024, the compliance publication TheCorporateCounsel.net warned EDGAR filers to check internal links before submission and to revisit older filings for broken internal links, framing link integrity as a filing-quality issue rather than a web nicety. That matters because the SEC's EDGAR full-text search still provides access to electronic filings dating back to 2001, so a link inside a disclosure document can be followed by a regulator or investor a decade after it was filed. A dead cross-reference in a live filing is a durable defect, not a transient one.

Second, healthcare price transparency. The Centers for Medicare and Medicaid Services requires hospitals to post a machine-readable file of standard charges at an accessible location, and it issues Notices of Violation and corrective-action requests to facilities that fall short of the hospital price transparency rules. An inaccessible or broken link to that required file is one of the ways a hospital can fall out of compliance, which turns a simple web maintenance task into a regulatory exposure with financial penalties attached.

Third, durable corporate records. Public registries such as the UK's Companies House keep filing histories that remain time-stamped and accessible for years, so the stable document paths behind those records are exactly what makes long-lived compliance possible. The lesson across all three is the same: when a link is part of a legal or regulatory record, its uptime is a control, not a convenience, and broken-link monitoring belongs on the compliance checklist as much as the SEO one.

The complete working project: an async link monitor

Here is a single script that ties the whole guide together. It reads your sitemap, checks every listed URL concurrently, writes a CSV report, prints the broken links, and exits with a non-zero status code when it finds any. That last detail matters: it lets you drop the script straight into a continuous-integration pipeline or a nightly cron job and fail the build automatically when a broken link ships. It uses httpx for asynchronous requests, which checks dozens of URLs at once instead of one at a time, and it falls back from HEAD to GET for servers that reject header-only requests.

Save it as link_monitor.py, install the one dependency with pip install httpx, edit the SITEMAP constant to point at your own sitemap, and run it with python link_monitor.py. Tune CONCURRENCY down if you see 429 responses, which mean you are hitting the server too fast.

# link_monitor.py
# Async broken-link monitor. Reads a sitemap, checks every URL,
# writes a CSV, and exits non-zero if anything is broken (for CI).
# Requires: Python 3.12+, httpx 0.27+   (pip install httpx)
import asyncio, csv, sys
import httpx

SITEMAP = '
.xml'
CONCURRENCY = 20
TIMEOUT = 15
UA = 'LinkAudit/1.0 (+
)'

async def fetch_sitemap(client):
    r = await client.get(SITEMAP)
    r.raise_for_status()
    urls = []
    for chunk in r.text.split('<loc>')[1:]:
        urls.append(chunk.split('</loc>')[0].strip())
    return urls

async def check(client, url, sem):
    async with sem:
        try:
            r = await client.head(url, follow_redirects=True, timeout=TIMEOUT)
            if r.status_code == 405:  # server rejected HEAD, retry GET
                r = await client.get(url, follow_redirects=True, timeout=TIMEOUT)
            return url, r.status_code, len(r.history)
        except httpx.HTTPError:
            return url, 0, 0

async def main():
    sem = asyncio.Semaphore(CONCURRENCY)
    async with httpx.AsyncClient(headers={'User-Agent': UA}) as client:
        urls = await fetch_sitemap(client)
        results = await asyncio.gather(*(check(client, u, sem) for u in urls))
    broken = [row for row in results if row[1] == 0 or row[1] >= 400]
    with open('link_report.csv', 'w', newline='') as f:
        w = csv.writer(f)
        w.writerow(['url', 'status', 'redirect_hops'])
        w.writerows(results)
    print(f'{len(urls)} checked, {len(broken)} broken')
    for url, code, hops in broken:
        print(f'  {code}  {url}')
    sys.exit(1 if broken else 0)

if __name__ == '__main__':
    asyncio.run(main())

The output on a healthy site ends with a line such as "482 checked, 0 broken" and exits 0. On a site with problems it prints each broken URL with its status code and exits 1, which fails a CI step automatically. Because it writes link_report.csv on every run, you also get a historical record you can diff over time to see whether your broken-link count is trending up or down. Point it at a staging sitemap in your deploy pipeline and no broken internal link will reach production without someone seeing it first. Extend it by reading the CSV into a dashboard, or by posting the broken list to a Slack webhook so the alert lands where your team already works.

What to do Monday morning

You do not need the whole program running to make progress this week. The fastest path to value is to prove the problem, fix the highest-value breaks, and set up one piece of monitoring so the problem does not silently return. Here is the concrete order of operations for your first few hours.

Start by crawling your site with the free edition of Screaming Frog, or a browser scanner if your site is under a few hundred pages, and filter the Response Codes for 4xx and 5xx. That single step gives you a real broken-link count and turns an abstract worry into a specific list. Export the Inlinks so you know which pages to edit. In parallel, open Google Search Console and read the Pages report for Not found (404) and Soft 404, which surfaces the dead URLs that external links reach and your crawl might miss.

Next, triage by value. Pull your 404 backlink report from Ahrefs or Semrush and 301 the dead URLs that still have referring domains, because reclaiming that equity is the highest-ROI move available and it takes minutes per URL. Then fix internal links: 301 the pages that moved, remove links to content that is genuinely gone, and correct the typos. Recrawl the affected URLs to confirm each one returns a 200 in a single hop, because an unverified fix is not a fix.

Finally, install the async monitor from the previous section, point it at your sitemap, and schedule it to run nightly with an alert to Slack or email. That converts broken-link management from a periodic fire drill into a background control. If broken links are hurting a revenue-critical funnel, or you want inbound equity reclaimed at scale, our link building and SEO teams do this as managed work. Either way, the sequence is the same: measure, fix the valuable breaks first, verify, and monitor so the web's natural decay never catches you by surprise again.

Alex Bain

Head of Growth

Alex leads programmatic SEO and AEO at Skitrate. Twelve years in growth, previously head of organic at a B2B SaaS unicorn and an ex-agency SEO director. Specializes in scaling content from zero to seven-figure traffic on technical and high-intent commercial verticals.

• 12 years leading SEO and growth at agencies and in-house SaaS
• Built and scaled programmatic SEO catalogs across SaaS, fintech, edtech
• Active practitioner of AEO/AIO/GEO measurement and citation tracking
• Public talks at SearchLove, BrightonSEO, MozCon

More posts by Alex Bain →