NGINX, X-Forwarded-For and the realip module
There’s a bug in NGINX that affects how the X-Forwarded-For
header is populated when the
ngx_http_realip_module is used;
the bug has been there for a long time and doesn’t look like it’s going to be fixed
anytime soon, by the look of this bug report.
The bug manifests itself when you have two or more proxies in front of your backend, for example when you use Cloudflare; say for example you have proxy1 (10.20.30.2) and proxy2 (10.20.30.3) having the following configuration:
proxy1 (10.20.30.2)
http {
server {
server_name _;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://10.20.30.3;
}
}
}
proxy2 (10.20.30.3)
http {
set_real_ip_from 10.20.30.2/32;
real_ip_recursive on;
real_ip_header X-Forwarded-For;
server {
server_name _;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://backend;
}
}
}
and you serve a request for a client (1.2.3.4); you would expect that the backend would receive the following headers:
X-Forwarded-For: 1.2.3.4, 10.20.30.2
X-Real-IP: 1.2.3.4
But in reality it will receive these:
X-Forwarded-For: 1.2.3.4, 1.2.3.4
X-Real-IP: 1.2.3.4
I think this happens because by the time NGINX give a value to $proxy_add_x_forwarded_for
it
has already replaced $remote_addr
with the original client’s IP.
One solution I’ve found is to use a tiny bit of Lua to create the correct version of
X-Forwarded-For
in the intermediate proxy (proxy2) and execute it in the right phase; in my
experiments I’ve found that the “access” phase is the only one that works correctly.
The configuration for proxy2 now looks like this:
http {
set_real_ip_from 10.20.30.2/32;
real_ip_recursive on;
real_ip_header X-Forwarded-For;
map $request $realip_add_x_forwarded_for { default ""; }
access_by_lua_block {
require("realip-x-forwarded-for").run()
}
server {
server_name _;
location / {
proxy_set_header X-Forwarded-For $realip_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://backend;
}
}
}
And the Lua script:
local _M = {}
function _M.run()
if (ngx.var.http_x_forwarded_for == "" or ngx.var.http_x_forwarded_for == nil) then
ngx.var.realip_add_x_forwarded_for = ngx.var.realip_remote_addr
else
ngx.var.realip_add_x_forwarded_for = ngx.var.http_x_forwarded_for .. ", " .. ngx.var.realip_remote_addr
end
end
return _M
Note how we have to give a default value to $realip_add_x_forwarded_for
using a
map, because nginx won’t allow set
statements in the http
level.
Demo:
$ curl http://proxy1/
-> X-Forwarded-For: 1.2.3.4, 10.20.30.2
-> X-Real-Ip: 1.2.3.4
$ curl http://proxy2/
-> X-Forwarded-For: 1.2.3.4
-> X-Real-Ip: 1.2.3.4
$ curl -H "X-Forwarded-For: 8.8.8.8" http://proxy1
-> X-Forwarded-For: 8.8.8.8, 1.2.3.4, 10.20.30.2
-> X-Real-Ip: 1.2.3.4
$ curl -H "X-Forwarded-For: 8.8.8.8" -H "X-Forwarded-For: 9.9.9.9" http://proxy1
-> X-Forwarded-For: 8.8.8.8, 9.9.9.9, 1.2.3.4, 10.20.30.2
-> X-Real-Ip: 1.2.3.4
You can experiment with this setup yourself with my nginx-lab if you checkout this PR.