How to Block Ahrefs, Moz and Majestic Bots in 2026

Reading time: 19 min

Ahrefs combed through roughly 461 million robots.txt files asking one question: which SEO crawler gets blocked the most? The answer isn’t the one most people guess.

When you count only explicit, targeted disallows, MJ12bot — Majestic’s crawler — is the single most specifically targeted SEO user-agent on the web. AhrefsBot, the fastest crawler in the entire SEO space, is shut out by 4.67% of the domains in that dataset.

So a lot of site owners are trying to block Ahrefs and Moz and Majestic at the door. Very few of them can tell you what the block actually bought them.

I’ve done both. I’ve blocked AhrefsBot, Dotbot and MJ12bot on sites where crawler traffic was eating more than half the monthly bandwidth bill. I’ve also reverted those blocks a year later, when the reporting damage finally outweighed the savings. This article gives you the rules, the 2026 blocking data and the trade-offs in one place — because nobody puts all three in the same page.

What Are Ahrefs, Moz and Majestic Bots — and Why Do They Keep Crawling You?

Three crawlers generate almost every complaint I get from site owners about “excessive bot traffic.” They are not malware. They are not scrapers. They are commercial indexers doing exactly what they were built to do.

Each one indexes publicly reachable URLs. If a page renders for an anonymous visitor, it renders for them. That’s the whole mechanism. There is no secret handshake.

AhrefsBot: the fastest crawler in the SEO space

AhrefsBot feeds the Ahrefs backlink index and also powers Yep.com, Ahrefs’ own search engine. That second part matters more than people realize — it’s why blocking AhrefsBot can have a visibility consequence beyond your reports.

Based on Cloudflare Radar bot request data, Ahrefs is by far the fastest crawler in the SEO space — roughly 4.6x faster than Moz’s Dotbot and about 6.7x faster than SemrushBot. That speed is exactly why bandwidth-heavy sites feel AhrefsBot first.

Dotbot: the engine behind Moz’s Domain Authority

Dotbot builds Moz’s link index. That index is the raw material for Domain Authority, Spam Score and the link explorer most agencies still show clients. It crawls slower than AhrefsBot but wider than people expect.

Block Dotbot and you don’t lose Domain Authority. You lose freshness. New pages take longer to be discovered, and Moz’s picture of your site drifts toward yesterday.

MJ12bot: Majestic’s link intelligence workhorse

MJ12bot powers Majestic’s Trust Flow and Citation Flow. It is the most explicitly targeted bot in the Ahrefs dataset — and I’ve seen this play out before: the older, more established link-intelligence vendors attract the most defensive blocking, usually from sites that have been around long enough to remember when link data was the entire game.

Majestic still discovers a large share of its links through Google’s index rather than through its own crawl. That’s the detail that complicates every “block them all” argument. The next section is the one most guides skip.

Bot name	User-agent token	Company	Primary purpose	Approximate crawl aggression	What you lose by blocking it
AhrefsBot	AhrefsBot	Ahrefs	Backlink index + Yep.com search engine	Highest in the SEO space (~4.6x Dotbot)	Yep.com visibility, page-history and internal-link data inside your own Ahrefs account
Dotbot	Dotbot	Moz	Link index behind Domain Authority	Moderate	Freshness of DA/Spam Score updates and new-page discovery
MJ12bot	MJ12bot	Majestic	Trust Flow / Citation Flow link intelligence	Moderate to high	Slower page-level link verification in Majestic reports
SemrushBot	SemrushBot	Semrush	Backlink index + site audit data	Lowest of the four	Freshness in Semrush backlink and audit reporting

Should You Actually Block Them? The Honest Cost-Benefit Analysis

Here’s the uncomfortable part. The most common reason people give for blocking SEO crawlers — “so competitors can’t see my links” — is the one reason that doesn’t work.

Your backlink profile is already in the vendor’s index. It persists whether or not AhrefsBot requests another byte from your server. And most link discovery routes through Google’s index, not through these crawlers. If a page is indexed by Google and links to you, Ahrefs, Moz and Majestic will discover that link through their own Google-fed pipelines regardless of your robots.txt.

So what does blocking actually buy you? Resource protection. That’s the real, measurable, defensible reason — and it’s the frame this entire decision should sit inside.

When blocking makes sense: crawl saturation and server cost

Last year I audited a publisher running 40,000+ AhrefsBot hits per month on a shared origin. Total crawler traffic — AhrefsBot, MJ12bot, Dotbot and assorted spoofers — accounted for 58% of monthly bandwidth and a visible chunk of origin CPU during peak editorial hours.

That’s a legitimate problem. When crawl saturation starts degrading real user response times, blocking is not secrecy — it’s capacity management. Nobody talks about this part, and it’s the only part that shows up in your hosting invoice.

When blocking backfires: visibility tools and client reporting

The same publisher discovered something else 60 days in. Their page-history graphs inside Ahrefs went flat. Internal-link data on new sections stopped populating. And because AhrefsBot also feeds Yep.com, a small but growing slice of referral traffic quietly vanished.

Rank tracking still worked, by the way. It reads Google’s SERPs, not your server. That’s the nuance most guides miss: you keep the output you buy the tool for, but you lose the diagnostic layer underneath it.

The myth that blocking hides your backlink profile from competitors

Any competitor willing to spend $129/month downloaded your link profile long before you considered a disallow rule. Blocking today changes nothing about what they already have. If a new link gets built tomorrow on a page Google indexes, they’ll see that too.

This isn’t a take — it’s a pattern. Every “hide your backlinks” tactic I’ve tested since 2009 has failed against the same mechanism: third-party discovery through search engine indexes.

Site type	Block all three?	Primary benefit	Primary risk	Recommended action
Large content publisher / news	Usually yes	Bandwidth and origin CPU savings; crawl saturation relief	Flat page-history data; minor Yep.com visibility loss	Block at server + WAF level; keep one monitoring bot if you use it daily
E-commerce at scale (100k+ SKUs)	Yes for AhrefsBot, selective for others	Removes repeat crawling of faceted and filtered URLs	Loses competitive merchandising intel from your own account	Block AhrefsBot and BLEXBot; leave Dotbot for now
SaaS marketing site	No	Negligible — traffic volume is tiny	Loses page-history evidence during technical migrations	Leave open; fix crawl waste instead
Agency (client reporting)	No	None — you sell the data these bots produce	Client-facing DA, DR and TF metrics go stale	Only block on client sites that explicitly ask, in writing
Local business (under 500 pages)	No	None — server load is irrelevant at this scale	Guarantees no benefit, adds reconciliation work	Do nothing

That last row is the honest one. If you run 300 pages on a decent host, blocking SEO crawlers is a hobby, not an optimization.

So blocking is not a yes/no. It’s a per-bot, per-site decision — and the next section shows what the rest of the web has actually chosen.

How Many Sites Actually Block These Bots? The 2026 Data

Let me show you the data instead of the opinion.

According to an Ahrefs study by Patrick Stox and data scientist Xibeijia Guan, published in 2025, the analysis covered approximately 140 million websites to determine how often SEO bots are blocked. For full coverage, the researchers parsed approximately 461 million robots.txt files.

Blocking rates by bot across 461 million robots.txt files

Across that corpus, the most blocked SEO bots were SemrushBot at 5.76%, Dotbot (Moz) at 5.34%, MJ12bot (Majestic) at 4.96%, BLEXBot at 4.88% and AhrefsBot at 4.67% (Ahrefs, 2025).

Note the distinction the study draws. Those percentages include every disallow that catches a bot. When you count only explicit targeted disallows — a rule written specifically naming that user-agent — Majestic’s MJ12bot is the most targeted, followed by Moz’s Dotbot (Ahrefs, 2025).

That gap is meaningful. SemrushBot gets caught incidentally more often. MJ12bot gets hunted on purpose.

Which sites block most aggressively: the DR>45 pattern

Among the top 1 million sites by DR — those above DR 45 — the pattern flips. Semrush is the most blocked, followed by Majestic and then Moz (Ahrefs, 2025).

Larger, more resource-rich sites block more aggressively. They have the engineering capacity to write firewall rules, the traffic volume to justify them, and the confidence that their own reporting won’t collapse. Smaller sites overwhelmingly leave the door open — which is the correct default for most of them.

Which website categories block the most

The category breakdown is where this gets genuinely interesting. Autos and Vehicles leads at 39%, followed by Books and Literature at 27% and Real Estate at 17% (Ahrefs, 2025).

Autos and Real Estate both run huge inventory sites with thousands of near-duplicate listing pages — exactly the architecture that attracts crawl saturation. Books and Literature is the outlier, and my read is that it’s a copyright reflex: publishers blocking anything that looks like bulk content collection.

Bot	Company	Disallow rate (%)	Explicitly targeted?	Rank in DR>45 cohort
SemrushBot	Semrush	5.76%	No — mostly incidental	1st
Dotbot	Moz	5.34%	Yes — 2nd most targeted	3rd
MJ12bot	Majestic	4.96%	Yes — most targeted overall	2nd
BLEXBot	WebMeUp / Semrush	4.88%	Partially	Not ranked separately
AhrefsBot	Ahrefs	4.67%	Yes	Not in top 3

Now the practical part. Three methods, in ascending order of how well they actually hold.

Method 1 — robots.txt: Fast, Polite, and Easy to Ignore

To block Ahrefs, Moz and Majestic crawlers in robots.txt, add a User-agent line for each bot followed by Disallow: /. Use the exact tokens AhrefsBot, MJ12bot, Dotbot and SemrushBot, each in its own block, and always allow Googlebot. Remember that robots.txt is a voluntary request, so pair it with server-level or firewall rules for a block that holds.

The exact user-agent tokens you need

Tokens are case-sensitive in practice and vendor-documented. Here’s the list:

• AhrefsBot — Ahrefs
• MJ12bot — Majestic
• Dotbot — Moz
• SemrushBot — Semrush
• BLEXBot — WebMeUp legacy / Semrush network

A complete copy-paste robots.txt block

User-agent: AhrefsBot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: Dotbot
Disallow: /

User-agent: SemrushBot
Disallow: /

User-agent: BLEXBot
Disallow: /

User-agent: Googlebot
Allow: /

User-agent: *
Disallow:

That final wildcard block keeps everything else open. Do not accidentally write Disallow: / under the wildcard unless you genuinely intend to shut the entire site out of every crawler on earth, including Google.

Five robots.txt mistakes that silently break the block

1. Casing errors. Ahrefsbot with a lowercase b does not match AhrefsBot. Vendor tokens are exact.
2. Missing trailing slash. Disallow: with nothing after it means “allow everything.” It must be Disallow: /.
3. Stacking multiple user-agents under one block. Each User-agent line needs its own group. Two consecutive User-agent lines apply only to the last one.
4. Directives placed after a wildcard Disallow. Order matters in some parser implementations. Put specific bot blocks above the wildcard.
5. A syntax error anywhere voids the parse. Most crawlers read the entire file; an invalid line early can invalidate later rules.

robots.txt is step one of three. It’s fast, it’s polite, and it’s easy to ignore — which is why the next method is the one that actually changes your server logs.

Method 2 — Server-Level Blocking with .htaccess and Nginx

This is where the block becomes real. Instead of asking a crawler to leave, you refuse to serve it — and you return a status code that says so.

Apache .htaccess user-agent blocking rules

Drop this at the top of your .htaccess file, above the WordPress block if you’re on WordPress:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (AhrefsBot|MJ12bot|Dotbot|SemrushBot|BLEXBot) [NC]
RewriteRule .* - [F,L]

The [NC] flag makes the match case-insensitive, which is safer than matching exact vendor casing. The [F,L] pair returns 403 Forbidden and stops rule processing immediately.

Nginx user-agent blocking rules

Inside your server block, before the location directives:

if ($http_user_agent ~* (AhrefsBot|MJ12bot|Dotbot|SemrushBot|BLEXBot)) {
    return 403;
}

Yes, “if is evil” in Nginx. But return inside an if at server level is one of the documented safe exceptions, and it’s the pattern every CDN vendor recommends. Don’t overthink it — just keep it out of location blocks.

The PHP header snippet — when it works and when it destroys your site

You’ll find snippets that read the $_SERVER['HTTP_USER_AGENT'] and call header('HTTP/1.1 403 Forbidden') inside a WordPress theme file. They work — until they don’t.

The failure mode is boring and predictable. Someone pastes the snippet into header.php without a child theme. A theme update six weeks later overwrites the file. The block silently disappears, nobody notices, and crawl traffic returns to baseline for three months before anyone checks a graph.

Use PHP header snippets only as a last resort, and only in a plugin or a child theme. Server-level rules should always sit on top of application-level ones.

403 vs 429 vs 503: choosing the right status code

• 403 Forbidden — the correct signal for a permanent refusal. “You are not permitted here.” Use this for crawlers you never want back.
• 429 Too Many Requests — the right signal for rate limiting. “Come back later, slower.” Use this when you want the data collected but throttled.
• 503 Service Unavailable — the wrong choice. 503 implies temporary outage and tells crawlers to retry. You are training the bot to come back harder.

Bot	robots.txt line	Apache .htaccess condition	Nginx directive	Expected status code
AhrefsBot	User-agent: AhrefsBot / Disallow: /	RewriteCond %{HTTP_USER_AGENT} AhrefsBot [NC]	if ($http_user_agent ~* AhrefsBot) { return 403; }	403
Dotbot (Moz)	User-agent: Dotbot / Disallow: /	RewriteCond %{HTTP_USER_AGENT} Dotbot [NC]	if ($http_user_agent ~* Dotbot) { return 403; }	403
MJ12bot (Majestic)	User-agent: MJ12bot / Disallow: /	RewriteCond %{HTTP_USER_AGENT} MJ12bot [NC]	if ($http_user_agent ~* MJ12bot) { return 403; }	403
SemrushBot	User-agent: SemrushBot / Disallow: /	RewriteCond %{HTTP_USER_AGENT} SemrushBot [NC]	if ($http_user_agent ~* SemrushBot) { return 403; }	403
BLEXBot	User-agent: BLEXBot / Disallow: /	RewriteCond %{HTTP_USER_AGENT} BLEXBot [NC]	if ($http_user_agent ~* BLEXBot) { return 403; }	403

Server rules stop the traffic at your origin. The next method stops it before it ever gets there.

Method 3 — Cloudflare WAF and Firewall Rules That Actually Hold

Edge blocking is the most robust option, and it’s the only one that solves the problem you started with. Traffic is refused before it reaches your origin, so the bandwidth and CPU you were trying to reclaim never get spent in the first place.

Building a Cloudflare WAF custom rule step by step

1. Open your Cloudflare dashboard and go to Security → WAF → Custom rules.
2. Create a new rule. Name it something you’ll recognize in six months, like “SEO crawler block.”
3. Set the expression to match the user-agent string.
4. Set the action to Block.
5. Save it in log-only mode first if your plan supports it, then promote to Block after 72 hours of clean logs.

The expression itself:

(http.user_agent contains "AhrefsBot") or (http.user_agent contains "MJ12bot") or (http.user_agent contains "Dotbot") or (http.user_agent contains "SemrushBot") or (http.user_agent contains "BLEXBot")

That’s it. The rule fires at the edge, and your origin nginx or Apache instance never sees the request.

Handling user-agent spoofing with IP ranges and reverse DNS

Here’s the honest limitation: user-agent strings are trivially spoofed. Anyone with curl can claim to be AhrefsBot. A user-agent block stops the polite crawler; it does not stop a determined scraper wearing that crawler’s name.

For spoof-proof blocking you need reverse DNS verification or IP-range matching against ranges the vendor publishes. Ahrefs, Moz and Majestic all publish crawl ranges. Verify each incoming request’s forward-confirmed reverse DNS against the claimed identity, and drop anything that fails.

The practical compromise most teams land on: user-agent rules for volume reduction, plus a monitoring alert for requests claiming OpenAI or Googlebot identities from non-verified IPs. You don’t need perfect enforcement. You need to stop the 40,000 automated hits a month.

Rate limiting versus a hard block

If your problem is throughput rather than presence, rate limiting is often the smarter play. A rule that allows AhrefsBot 20 requests per minute and returns 429 beyond that keeps your site discoverable while capping the bandwidth draw.

I’ve used this on sites where the marketing team relied on the tool daily. It’s the option nobody talks about, and it’s usually the right one.

Three methods, three layers. Now let’s get specific about each crawler, because most readers only care about one of them.

Blocking Each Bot Individually: AhrefsBot, Dotbot, MJ12bot and SemrushBot

Not everyone wants to block all four. Here’s the per-bot breakdown, including the fourth bot readers always ask about once they see the blocking data.

AhrefsBot: the highest-volume crawler and what blocking costs you

AhrefsBot is by far the fastest crawler in the SEO space — roughly 4.6x faster than Moz’s Dotbot and about 6.7x faster than SemrushBot, based on Cloudflare Radar bot request data cited by Ahrefs (2025). Bandwidth-heavy sites feel it first because it simply requests more.

What breaks when you block it: your pages disappear from Yep.com, Ahrefs’ own search engine. Page-history graphs, internal-link reports and site-audit crawl data inside your own Ahrefs account also degrade, because those features depend on AhrefsBot having visited.

Dotbot: Moz Domain Authority and link index dependencies

Dotbot feeds Moz’s link index, which is the raw input for Domain Authority. Block it and DA doesn’t vanish — it stops refreshing. New page discovery slows, and your link profile drifts toward a stale snapshot.

If you show DA to clients, test the impact on a staging property before you commit. Agencies who block Dotbot on client sites usually regret it within a quarter.

MJ12bot: Majestic Trust Flow and Citation Flow

MJ12bot is the most explicitly targeted SEO crawler on the web (Ahrefs, 2025), which tells you something about how defensive established webmasters get about link-data vendors. Blocking it lets your existing Trust Flow and Citation Flow scores persist, but fresh page-level discovery and link verification slow noticeably.

Since Majestic also discovers links through Google’s index, blocking MJ12bot does not remove your backlinks from Majestic. It just changes how quickly new ones appear.

SemrushBot: the most blocked SEO bot in the dataset

SemrushBot tops the overall blocking chart at 5.76% (Ahrefs, 2025), and it’s also the most blocked among DR>45 sites. Interestingly, most of those disallows are incidental rather than targeted — it gets caught by broad rules rather than named specifically.

You can block AhrefsBot while keeping SemrushBot, and plenty of teams do exactly that. SemrushBot is the slowest of the four, so it costs you the least in server resources and delivers site-audit value that many technical SEO teams rely on weekly.

Bot	What it powers	Relative crawl speed	What breaks when blocked	Minimal blocking rule
AhrefsBot	Ahrefs backlink index + Yep.com	Fastest (~4.6x Dotbot)	Yep.com visibility; Ahrefs page-history and internal-link data	User-agent: AhrefsBot / Disallow: /
Dotbot	Moz link index + Domain Authority	Moderate	Freshness of DA, Spam Score and new-page discovery	User-agent: Dotbot / Disallow: /
MJ12bot	Majestic Trust Flow / Citation Flow	Moderate to high	Speed of page-level link verification in Majestic	User-agent: MJ12bot / Disallow: /
SemrushBot	Semrush backlink index + site audit	Slowest (~1/6.7 of Ahrefs)	Freshness in Semrush backlink and audit reporting	User-agent: SemrushBot / Disallow: /
BLEXBot	Legacy WebMeUp / Semrush network	Low	Minor — limited active reporting dependency	User-agent: BLEXBot / Disallow: /

Rules deployed. Now prove they’re working — because a block you haven’t verified is a block you don’t have.

How to Verify the Block Works — and What to Do If It Doesn’t

Most guides end at implementation. Implementation is the midpoint. Verification is the part that decides whether you actually solved anything.

Reading your access logs to measure bot traffic

Get a baseline before you deploy. On nginx:

grep -i "AhrefsBot" /var/log/nginx/access.log | wc -l
grep -i "MJ12bot" /var/log/nginx/access.log | wc -l
grep -i "Dotbot" /var/log/nginx/access.log | wc -l

On Apache, the file is usually access.log in the same path or under /var/log/apache2/. Run these weekly, log the numbers in a spreadsheet, and keep the series going for 30 days after deployment.

You want to see a steep drop in week one and a plateau after. Not zero. Never zero.

Testing the block with curl and a spoofed user-agent

The fastest verification is a single command. Send a request claiming to be AhrefsBot and check the status line:

curl -A "AhrefsBot/7.0 (+http://ahrefs.com/robot/)" -I https://yourdomain.com/

You want to see HTTP/2 403 in the output. If you see 200, your rule isn’t firing — check that the WAF rule is above any allow rule, that the .htaccess file is being read, and that caching isn’t serving a stored response.

Then run the control test. Same command with a normal browser user-agent. That must return 200. If it returns 403, you’ve blocked real users and you need to roll back immediately.

And check Search Console crawl stats after deploy. Googlebot traffic should be flat. A rise in crawl errors there means you’ve caught Google in the rule — the single most expensive mistake in this whole process.

Why blocks never reduce traffic to zero

A successful block produces a plateau, not a floor. Three reasons. Spoofers keep hitting you wearing a blocked identity. Cached DNS routes and distributed crawl nodes mean some requests arrive from IPs you haven’t fully accounted for. And legacy queue backlog takes days to drain.

If your AhrefsBot hits drop from 40,000 to 900 per month, that’s a win. If they drop to zero, you’ve probably blocked something you didn’t mean to.

Test performed	Expected result	What a failure indicates	Fix
curl with spoofed AhrefsBot user-agent	HTTP 403	Rule not firing or cached response served	Move WAF rule above allow rules; purge cache; confirm .htaccess is read
curl with normal browser user-agent	HTTP 200	Over-broad rule catching real visitors	Narrow the expression; check for wildcard matches
grep log for blocked user-agents	Steep drop, then plateau	Zero hits = likely blocked too much; no drop = rule inactive	Re-scope the rule; verify at all three layers
Search Console crawl stats	Googlebot flat, errors flat	Rising crawl errors = Googlebot is being caught	Immediately whitelist Googlebot; re-test
Origin CPU / bandwidth over 7 and 30 days	Downward trend	Flat trend = block is not saving resources	Move blocking to the edge instead of the origin

Frequently asked questions

Does blocking AhrefsBot remove my backlinks from Ahrefs?

No. Blocking stops future crawling of your pages. It does not delete links already stored in the Ahrefs index. Links continue to be discovered through Google’s index and through links pointing at you from other sites, so your profile can still update.

Will blocking MJ12bot affect my Majestic Trust Flow or Citation Flow?

Your existing Trust Flow and Citation Flow scores remain intact. What slows down is fresh page-level discovery and link verification. If you rely on Majestic reporting for client work, test the impact on a single property before committing site-wide.

Does blocking MozBot stop Domain Authority from updating?

Domain Authority is calculated from Moz’s own link index, which Dotbot feeds. Blocking Dotbot limits new page discovery, which reduces the freshness of the metric rather than erasing it. Your clients won’t see DA disappear — they’ll see it stop moving.

Is robots.txt enough to block Ahrefs, Moz and Majestic?

No. robots.txt is a request, not enforcement. Use it as the first layer, then add server-level .htaccess or Nginx rules and an edge WAF rule so that refusing traffic no longer depends on crawler goodwill.

Can SEO crawlers bypass a user-agent block?

Yes. User-agent strings are easily spoofed. Pair them with IP-range allowlisting or reverse DNS verification against the vendor’s published ranges, and monitor your logs for traffic claiming a blocked identity from unverified IPs.

Does blocking these crawlers hurt my rankings in Google?

Google does not use Ahrefs, Moz or Majestic data as a ranking signal. Blocking them cannot directly lower your Google rankings — provided you never accidentally disallow Googlebot or overwrite the robots.txt rules you need.

Will blocking SEO bots reduce my server load and hosting costs?

Often, yes — especially with AhrefsBot, the fastest crawler in the SEO space at roughly 4.6x Moz’s crawl rate. Measure your access logs before and after to quantify the bandwidth and CPU you actually recover, and be honest if the number is small.

The Decision You’re Actually Making

Four things worth carrying out of this.

• robots.txt alone is a request, not enforcement. Layer robots.txt, server rules and a WAF if you want a block that survives a crawler choosing to ignore you.
• Blocking protects server resources and crawl budget. It does not hide your backlink profile from competitors who already downloaded it.
• Know the exact tokens and verify the result. AhrefsBot, Dotbot, MJ12bot and SemrushBot — plus a curl test confirming 403 and a log check confirming the drop.
• The trade-off is real. Blocking AhrefsBot removes you from Yep.com results and degrades your own page-history and internal-link data. That cost doesn’t appear on your hosting invoice, but it’s still a cost.

Slow down. Think.

Before you paste that disallow rule, ask yourself a blunt question: are you protecting your site, or blinding the tools you use to measure it?

If the honest answer is server load, then measure your logs before and after. Quantify the bandwidth you reclaimed, the CPU you freed, the crawl saturation you killed. Make the block pay for itself in numbers you can show someone. That’s the only version of blocking AhrefsBot, Dotbot and MJ12bot that holds up a year from now — when the next algorithm update lands and you need six months of clean history to understand what happened.

The playbook changed. Again. But the underlying rule hasn’t moved since 1997: know what you’re refusing before you refuse it.