All I needed to do was add

if(isset($_SERVER['HTTP_X_FORWARDED_FOR'])) {
        $list = explode(',',$_SERVER['HTTP_X_FORWARDED_FOR']);
        $_SERVER['REMOTE_ADDR'] = $list[0];
  }

to wp-config.php. This bit checks if HTTP_X_FORWARDED_FOR is set in the HTTP request, and if so, uses it for the REMOTE_ADDR.

Update:

I found a better way to do this, assuming that the backend server is running apache2. Just install mod_rpaf. On Debian:

apt-get install libapache2-mod-rpaf

Then edit /etc/apache2/mods-enabled/rpaf.conf and set your proxy IP. This also allows apache’s access.log to show the real client IP.

One minor bug (not sure if it’s because of this code or lighttpd or my network setup) is that all the proxied IPs are prefixed with ::ffff:. In any case it’s just a minor annoyance.