Mortise 4 min read

Why broken-link checkers keep reporting the same broken links

A one-off crawl has no memory, so yesterday's 404 is today's finding. A baseline stores the known breaks and alerts only when a working link newly fails.

They report the same links because each crawl starts from zero. The tool finds every URL that 404s, including the ones it found last time, and it has no idea you already decided to leave them. The fix is a baseline: remember what was already broken, and speak up only when a link that used to work newly fails.

One-off audits

A broken-link crawl is a useful afternoon. You export a list, fix the internal links you still control, and shrug at the external ones that died years ago. The list did its job.

Run the same crawl next month and the shrug comes back as if it were news. The old campaign URL is still dead. The PDF a partner removed in 2022 is still gone. The footer link you meant to delete is still there, still broken, still at the top of the report because the path is short.

Nothing is wrong with the crawler. It was asked "what is broken right now?" and it answered. You needed "what broke since last time?"

Alert fatigue is a product problem

If every report repeats the same twenty rows, people stop opening the report. The twenty-first row, the one from this week's template change, sits in the same file and gets the same ignore.

Fatigue here is specific. It is not "too many emails" in the abstract. It is a list that cannot tell a new failure from a stored one, delivered often enough that the new failure has nowhere to stand out.

Two habits make it worse:

  • Scheduling a full audit and sending the entire result every time.
  • Treating "we emailed you" as success, even when the email is the same email as last week.

What baseline-first means

Mortise separates those two questions.

You verify a site you own. The first crawl is silent. It stores the state of the links it saw, and it does not alert. Links that were already 404 or 410 on that crawl stay quiet afterwards.

Later crawls look for a change in that state. A link alerts only after it fails twice in a row as 404 or 410, and only if it had been working. One failed response is not enough. A timeout is not a broken link. A page that was already broken is not a new break.

That is a narrower promise than "we will list every broken URL on your site, forever." It is the promise that makes a scheduled check bearable.

Free plan: 1 verified site, 200 pages per crawl, manual crawl at most once every 24 hours, results on the dashboard, no card. A failed crawl does not consume the daily attempt. Pro is $12/month or $79/year: 10 sites, 2,000 pages, a daily or weekly schedule, email and a signed webhook for new breaks only, and CSV. New-break history is 14 days on Free and 180 days on Pro.

www and the apex count as one site. External links found on the pages it crawls are checked too.

When you still want the full list

Sometimes you do want every current 404, including the old ones. A launch, a content audit, or a cleanup day is that moment. Look at the crawl itself and fix what you came to fix. Do not turn that full list into a daily alert.

For a plain-English punch list that also covers HTTP errors, redirects, SEO basics, and SSL — not only links — run SiteCheck. Free: one-off report, up to 25 pages, no account. How that job differs from Mortise is in website inspection vs new-break alerts.

After the cleanup, take a fresh baseline if the "known broken" set has changed a lot. From then on, the alert is the regression.

A migration is the usual time this matters. The order of work is in how to find newly broken links after a website migration.

What people do instead

  • Bookmark a crawler and re-run it when someone complains. The broken link has already had visitors.
  • Export CSV from a desktop suite and diff the files by hand. That works, and it is a chore you will skip.
  • Subscribe to an audit that emails the full list. The mail rule that archives it is the honest review of the product.

A baseline is the same crawl with a memory and a rule for when to interrupt you.

What it will not catch

It will not tell you that the words on a page changed. A pricing table can be rewritten and every link can still return 200. Text changes, downtime, and certificate expiry are SignalWatch. The two jobs side by side are in website change monitoring vs broken-link monitoring.

Checklist

  • You can say which links are allowed to stay broken. Those belong in the baseline, quiet.
  • The first crawl is not treated as an alert. Silence on day one is correct.
  • A link has to fail twice as 404 or 410 before it counts.
  • The ongoing check is aimed at new breaks, not at re-proving the old list.
  • A full inventory, when you need one, is a separate sitting, not the daily email.