The Real Client IP Behind a Proxy: X-Forwarded-For and a Rate Limit That Didn't Work

Adding a per-IP rate limit to a web app looks easy: take the address the request came from, count its requests per minute, return 429 once it goes over. When the app runs behind a proxy, that first step, "the address the request came from", is not as simple as it sounds. This post is about two rate limits in two of my projects that did not work on the live site. One never filled up; the other could be bypassed completely with a fake header. Both had the same root cause: which value of the X-Forwarded-For header to trust.

What "client address" means behind a proxy

Most of my server-side projects on free tiers run on Render, and Render sits behind Cloudflare. A visitor's request reaches the app like this:

visitor  →  Cloudflare server  →  Render proxy  →  app

The connection the app sees comes from Render's proxy, not from the visitor. Proxies carry the visitor's address in the X-Forwarded-For header. Each proxy appends the address it received the request from:

X-Forwarded-For: <whatever the visitor wrote>, <visitor>, <Cloudflare server>

That list has two properties:

The leftmost values are in the client's hands. A visitor can send their own X-Forwarded-For header. Proxies do not remove it; they only append to it.
Counting from the right gets you to trustworthy values, but only as many as there are known proxies. Without knowing which proxy wrote what, no value can be trusted.

Bug 1: the limit never filled up (Crypto Toolkit, ASP.NET Core)

Crypto Toolkit had a limit of 300 requests per minute per IP. To get the right address behind Render's proxy, I turned on ASP.NET Core's ForwardedHeaders middleware. I did not know the proxy's address in advance, so I cleared the known proxy lists:

builder.Services.Configure<ForwardedHeadersOptions>(options =>
{
    options.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
    options.KnownIPNetworks.Clear();
    options.KnownProxies.Clear();
});

With this setup the middleware takes the rightmost value as the client address. Everything worked locally and in the tests. In the final review before release I sent 320 parallel requests to the live site and did not get a single 429.

The reason: the rightmost value was not the visitor but the Cloudflare server that passed the request to Render. Cloudflare has many servers, and one visitor's requests can go through different ones. Every Cloudflare address opened its own counter, so no counter ever reached the limit. The rate limit existed in the code but not in production.

Bug 2: a fake header bypassed it (KVKK Document Assistant, Python)

KVKK Document Assistant is a FastAPI app served by uvicorn. So that the links it generated would use https, I let uvicorn read proxy headers and told it to trust every proxy:

uvicorn app.main:app --proxy-headers --forwarded-allow-ips='*'

With every address treated as a trusted proxy, uvicorn took the leftmost value as the client address, and that value was the one the visitor wrote. In the final review I sent requests to the live site, each with a different made-up X-Forwarded-For: despite a limit of 20 documents per 10 minutes, 22 out of 22 requests went through. Every request could pretend to be someone else.

The two bugs look like opposites: one trusted the rightmost value, the other the leftmost. What they share is that in both I picked a value without knowing who wrote which part of the list.

I measured before fixing

Instead of guessing which header carried what, I added a temporary diagnostic endpoint to the KVKK project. It showed every IP-related header as the request reached the app. Sending requests to the live site from my own connection, with and without a fake header, I saw:

X-Forwarded-For: the value I wrote on the left, then my real address, then a Cloudflare server on the right.
CF-Connecting-IP: always my real address.

Cloudflare writes the visitor's real address into CF-Connecting-IP itself. If a visitor tries to send that header, Cloudflare rejects the request with a 403; I tried that on the live site too. In this setup, it was the only value nobody could fake.

The fix

The rate limit reads CF-Connecting-IP, and falls back to the direct connection address when it is missing (local runs):

public static class ClientIp
{
    public const string CloudflareHeader = "CF-Connecting-IP";

    public static string Of(HttpContext context)
    {
        var header = context.Request.Headers[CloudflareHeader].ToString();
        if (IPAddress.TryParse(header, out var address)) return address.ToString();
        return context.Connection.RemoteIpAddress?.ToString() ?? "unknown";
    }
}

The Python side follows the same logic:

def client_ip(request: Request) -> str:
    cf = request.headers.get("cf-connecting-ip", "").strip()
    if cf:
        return cf
    return request.client.host if request.client else "unknown"

I kept uvicorn's --proxy-headers, because without it the links the app generated turned into http. That setting now only matters for the scheme; the rate limit never looks at X-Forwarded-For.

After the fix I repeated the same tests on the live site. On KVKK the fake headers no longer helped and the 21st request got a 429. I applied the same fix to Casus and Sahaf, which also run on Render behind Cloudflare, and tested Casus on the live site as well.

A different setup, a different right answer

CF-Connecting-IP is not the right answer everywhere, only where Cloudflare is in front. My cookieless analytics tool Nabız runs on my own server on Oracle Cloud, behind Caddy. There is no Cloudflare in front, and Caddy appends the address it received the request from. So Nabız has a different rule, and two different addresses:

// clientIP: Caddy arkasındaysak (-trust-proxy) X-Forwarded-For'un son değeri,
// yani Caddy'nin kendi eklediği adres; değilse doğrudan bağlantının adresi.
// İstemcinin kendi yazdığı sahte değerler listenin başında kalır, onlara bakmayız.

(Behind Caddy, the last value is the one Caddy added; the client's fake values stay at the start of the list and are ignored.)

The last value for the rate limit: the one Caddy appended, which nobody can fake.
The first value for counting unique visitors: when a site serves the Nabız script from its own domain, through Vercel for example, the last value is Vercel's address and every visitor would count as one person. The first value can be faked, but the worst outcome is a slightly inflated count; nobody's data leaks and the limit is not bypassed.

Zarf runs behind Caddy on the same machine. I put the Caddy setting that preserves X-Forwarded-For only in Nabız's block; writing it globally would have changed how Zarf reads addresses too.

What I took away

X-Forwarded-For is a list, and its left side belongs to the client. Settings that mean "trust every proxy" (--forwarded-allow-ips='*', KnownProxies.Clear()) make you pick the wrong value from it.
The right value depends on the setup. With Cloudflare in front, the header Cloudflare writes; with my own proxy in front, the last value that proxy appended. I don't write this code without knowing how many proxies there are and what each one writes.
This bug is invisible locally. There is no proxy on my machine, and the tests talk to the app directly. I found both bugs in the final review before release by sending requests to the live site. Since then, every project's final review tests the rate limit on the live site once with a fake header and once with parallel requests.