REST API Crawlers Were Burning 80% of a Client's Server CPU — Rate-Limiting /wp-json/ Without Breaking WordPress

A client's WooCommerce store had been running smoothly for months — sub-second page loads, stable CPU around 15-20% on a 4-core VPS. Then the monitoring alerts started. CPU hit 78%, then 85%, then stayed above 90% for three hours straight. The store was still responding, but pages that normally loaded in 800ms were taking 6-7 seconds. Checkout was timing out for some customers.

The usual suspects didn't pan out. No plugin updates had been applied. No traffic spike in Simple Analytics. WP-Cron was running on system cron, not causing pile-ups. Redis was connected and healthy.

I tailed the Nginx access log and immediately saw the problem.

Thousands of Requests to /wp-json/ From Automated Crawlers

grep "wp-json" /var/log/nginx/access.log | wc -l

Over 23,000 requests to /wp-json/ endpoints in the last four hours. I broke them down by endpoint:

grep "wp-json" /var/log/nginx/access.log \
  | awk '{print $7}' \
  | sed 's/\?.*//' \
  | sort | uniq -c | sort -rn | head -15

The output told the story:

   8412  /wp-json/wp/v2/posts
   4891  /wp-json/wp/v2/pages
   3204  /wp-json/wp/v2/categories
   2187  /wp-json/wp/v2/tags
   1876  /wp-json/wp/v2/comments
    943  /wp-json/wp/v2/users
    812  /wp-json/wp/v2/media
    401  /wp-json/oembed/1.0/embed
    294  /wp-json/

Bots were systematically crawling every public REST API endpoint. The /wp-json/wp/v2/posts endpoint was hit hardest — it returns paginated post content by default, and each request was triggering full WordPress bootstrap, database queries for post data, and serialisation of the response.

I checked the source IPs:

grep "wp-json" /var/log/nginx/access.log \
  | awk '{print $1}' | sort | uniq -c | sort -rn | head -10

A mix of cloud provider IPs (AWS, DigitalOcean, Hetzner) and residential proxies. Classic bot traffic — no single IP dominant enough to block with a simple firewall rule.

Why REST API Abuse Hits Harder Than You'd Expect

The difference between a bot crawling your HTML pages and a bot crawling /wp-json/ is what happens on the server.

When a bot requests a cached page, Nginx serves it from the FastCGI cache or a static file. PHP never runs. CPU cost: negligible.

When a bot requests /wp-json/wp/v2/posts, the request bypasses page cache entirely. REST API responses are dynamic by default. Each request:

1. Spawns a PHP-FPM worker
2. Bootstraps WordPress core (loading wp-load.php, all active plugins, the active theme)
3. Runs database queries to fetch posts, terms, meta, and author data
4. Serialises the response as JSON
5. Returns it to the bot, which immediately requests the next page

On this client's server, a single REST API request took 180-250ms of PHP-FPM worker time. With hundreds of concurrent bot requests, the PHP-FPM pool was permanently saturated. Legitimate customer requests had to queue behind bot requests for an available worker.

This is the same class of problem as XML-RPC brute force attacks — the damage comes from PHP-FPM worker exhaustion, not from the content being accessed.

Why You Can't Just Block /wp-json/ Entirely

Unlike xmlrpc.php, which can be safely blocked on most sites in 2026, the REST API is actively used by WordPress itself. The block editor (Gutenberg) relies on REST API endpoints to save posts, load block data, and manage media. WooCommerce uses REST API endpoints for its admin interface, order processing, and block-based checkout. Many plugins use the REST API for their settings pages and AJAX operations.

Blocking /wp-json/ entirely would break your own admin.

The fix has to be surgical: rate-limit bot traffic to the REST API without disrupting authenticated users or legitimate integrations.

The Fix: Nginx Rate Limiting for /wp-json/

I added a rate limiting zone and location block to the Nginx server configuration.

First, define a rate limiting zone in the http block (typically in /etc/nginx/nginx.conf or a shared conf file):

limit_req_zone $binary_remote_addr zone=restapi:10m rate=5r/s;

This creates a shared memory zone called restapi that tracks request rates per client IP. The 10m allocates 10MB of shared memory (enough for roughly 160,000 IP addresses). The rate=5r/s allows 5 requests per second per IP — more than enough for legitimate use, far too slow for a bot crawling systematically.

Then, in the server block, add a location directive for /wp-json/:

location /wp-json/ {
    limit_req zone=restapi burst=15 nodelay;
    limit_req_status 429;

    # Pass to PHP-FPM as normal
    try_files $uri $uri/ /index.php?$args;
    include fastcgi_params;
    fastcgi_pass unix:/run/php/php-fpm.sock;
    fastcgi_param SCRIPT_FILENAME $document_root/index.php;
}

The burst=15 allows short bursts of up to 15 requests beyond the rate limit before throttling kicks in. The nodelay processes burst requests immediately rather than queuing them. The limit_req_status 429 returns a proper "Too Many Requests" status code instead of the default 503.

Reload Nginx:

nginx -t && systemctl reload nginx

Exempting Logged-In Users From the Rate Limit

The rate limit above applies to everyone, including the site owner editing posts in the block editor. Gutenberg makes dozens of REST API requests during a single editing session — auto-saves, block previews, media queries. A blanket 5r/s limit will make the editor feel sluggish.

The fix is to use a map directive that sets the rate limit zone conditionally based on a WordPress auth cookie:

map $http_cookie $rate_limit_key {
    default        $binary_remote_addr;
    "~wordpress_logged_in_"  "";
}

limit_req_zone $rate_limit_key zone=restapi:10m rate=5r/s;

When a request includes a wordpress_logged_in_ cookie, $rate_limit_key is set to an empty string, which effectively exempts that request from rate limiting. Unauthenticated requests use the client IP as the key and get rate-limited normally.

This is an important nuance. Without this exemption, I'd be solving one problem (bot abuse) while creating another (broken editor for the client).

Adding a Fail2Ban Jail for Aggressive Crawlers

Rate limiting at Nginx level handles the throughput problem, but persistent bots will keep hammering the endpoint at the allowed rate indefinitely. For bots that hit the rate limit repeatedly, I add a Fail2Ban jail to ban the IP at the firewall level.

Create a filter at /etc/fail2ban/filter.d/nginx-restapi.conf:

[Definition]
failregex = ^<HOST> .* "(GET|POST) /wp-json/.* HTTP/.*" 429
ignoreregex =

Create the jail in /etc/fail2ban/jail.d/nginx-restapi.conf:

[nginx-restapi]
enabled  = true
port     = http,https
filter   = nginx-restapi
logpath  = /var/log/nginx/access.log
maxretry = 30
findtime = 60
bantime  = 3600

This bans any IP that triggers 30 rate-limit responses (429s) within 60 seconds for one hour. Legitimate users will never hit this threshold. Automated crawlers will hit it within minutes.

Restart Fail2Ban:

systemctl restart fail2ban

Verify the jail is active:

fail2ban-client status nginx-restapi

Disabling REST API Endpoints You Don't Need

Rate limiting reduces the damage, but you can shrink the attack surface further by disabling REST API endpoints that serve no purpose on your site. Most WooCommerce stores don't need the public /wp/v2/users endpoint (and you should already be blocking it to prevent user enumeration). Comments endpoints are often unnecessary if you've disabled comments. Tags and categories endpoints rarely need to be public.

Add this to your theme's functions.php or a custom mu-plugin:

add_filter('rest_endpoints', function ($endpoints) {
    if (is_user_logged_in()) {
        return $endpoints;
    }

    $routes_to_remove = [
        '/wp/v2/users',
        '/wp/v2/users/(?P<id>[\d]+)',
        '/wp/v2/comments',
        '/wp/v2/comments/(?P<id>[\d]+)',
    ];

    foreach ($routes_to_remove as $route) {
        if (isset($endpoints[$route])) {
            unset($endpoints[$route]);
        }
    }

    return $endpoints;
});

The is_user_logged_in() check ensures the block editor and WooCommerce admin still have full REST API access — endpoints are only removed for unauthenticated requests. Don't remove endpoints that WordPress or WooCommerce actively use on the public side. Test any changes on staging first.

The Result

After applying the Nginx rate limit and Fail2Ban jail, the effect was immediate. CPU dropped from 92% to 18% within ten minutes. PHP-FPM workers went from permanently saturated to mostly idle. Page load times returned to under a second.

Over the following 24 hours, Fail2Ban banned 87 IPs. The Nginx rate limiting returned 3,400 429 responses — requests that would previously have consumed PHP-FPM workers and database connections.

The client's block editor still worked perfectly. WooCommerce admin, checkout, and order processing were unaffected. The rate limit only constrained unauthenticated high-volume requests.

Monitoring Script

I now run this daily across managed servers to track REST API request volume:

#!/bin/bash
THRESHOLD=2000
LOG="/var/log/nginx/access.log"
COUNT=$(grep -c "wp-json" "$LOG" 2>/dev/null) || COUNT=0
if [ "$COUNT" -gt "$THRESHOLD" ]; then
    echo "wp-json hit $COUNT times today on $(hostname)" \
      | mail -s "REST API Traffic Alert" [email protected]
fi

A well-configured WordPress site should see very few unauthenticated REST API requests per day. If this alert fires, something is crawling you.

Prevention Checklist

• Rate-limit /wp-json/ at the Nginx level — don't rely on WordPress plugins, the request has already consumed a PHP-FPM worker by the time a plugin runs
• Exempt authenticated users using the cookie map technique so the block editor and WooCommerce admin aren't affected
• Add a Fail2Ban jail for IPs that persistently trigger rate limits
• Disable unused REST API endpoints to reduce the attack surface
• Monitor REST API request volume in access logs — sudden spikes indicate automated crawling
• Block the REST API user endpoint separately to prevent username enumeration

---

Stop Firefighting. Start Maintaining.

REST API abuse is one of the less obvious attack vectors I see across the 70+ WordPress sites I manage. Unlike a brute force attack on wp-login.php, there's no failed login notification. No security plugin alert. Just a steadily climbing CPU graph and customers complaining about slow pages.

If your server is running hot and you can't figure out why, I can help. Whether you need ongoing server management or a one-off performance investigation — that's what I do.

View Maintenance Plans → Get in Touch →