HAProxy is a great load balancer; however, it doesn't natively terminate SSL connections. Fortunately, stunnel can terminate SSL connections and seamlessly forward them to HAProxy. To be able to log the client's IP address, we'll need to patch stunnel to provide X-Forwarded-For headers to HAProxy.
The following instructions are based on Debian Squeeze, HAProxy 1.4.8-1, and stunnel 4.32.
To get started, download the stunnel source and X-Forwaded-For patch:
wget http://www.stunnel.org/download/stunnel/src/stunnel-4.32.tar.gz wget http://haproxy.1wt.eu/download/patches/stunnel-4.32-xforwarded-for.diff
Before building stunnel, we need to install a few other packages:
apt-get install build-essential libssl-dev haproxy
To install stunnel:
tar -zxvf stunnel-4.32.tar.gz cd stunnel-4.32 patch -p1 < ../stunnel-4.32-xforwarded-for.diff ./configure make && make install adduser stunnel ln -s /usr/local/etc/stunnel /etc/stunnel touch /var/log/stunnel.log chown -R stunnel:stunnel /usr/local/etc/stunnel chown -R stunnel:stunnel /var/run/stunnel chown stunnel:stunnel /var/log/stunnel.log
Then we have to create the necessary files in /etc/stunnel: stunnel.conf, stunnel.pem, and the SSL certificate (server.crt) and key file (server.key). The server.crt and server.key are the SSL certificate and key which would normally go on a web server. The following stunnel.conf will accept incoming HTTPS connections on port 443, then forward it to port 81 as regular HTTP traffic – just change the ip address from 192.168.1.1 to your own.
Example /etc/stunnel/stunnel.conf:
cert=/etc/stunnel/stunnel.pem setuid=stunnel setgid=stunnel pid=/var/run/stunnel/stunnel.pid output = /var/log/stunnel.log socket=l:TCP_NODELAY=1 socket=r:TCP_NODELAY=1 [https] cert=/etc/stunnel/server.crt key=/etc/stunnel/server.key accept=192.168.1.1:443 connect=192.168.1.1:81 xforwardedfor=yes
Next, we'll need to create our stunnel certificate file, stunnel.pem. More details about the certificate creation process are available at Using Certificates with stunnel, but you can just run the following command:
openssl req -new -x509 -days 365 -nodes -config /etc/stunnel/stunnel.conf -out /etc/stunnel/stunnel.pem -keyout /etc/stunnel/stunnel.pem
Finally, here's an example haproxy.cfg file (/etc/haproxy/haproxy.cfg) – just change your web server IPs:
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
user haproxy
group haproxy
daemon
maxconn 20000
defaults
log global
option dontlognull
balance leastconn
clitimeout 60000
srvtimeout 60000
contimeout 5000
retries 3
option redispatch
listen http 192.168.1.1:80
mode http
cookie WEBSERVERID insert
option httplog
balance source
option forwardfor except 192.168.1.1
option httpclose
option redispatch
maxconn 10000
reqadd X-Forwarded-Proto:\ http
server webserver1 192.168.1.2 cookie webserver1 maxconn 5000
server webserver2 192.168.1.3 cookie webserver2 maxconn 5000
listen https 192.168.1.1:81
mode http
cookie WEBSERVERID insert
option httplog
balance source
option forwardfor except 192.168.1.1
option httpclose
option redispatch
maxconn 10000
reqadd X-Forwarded-Proto:\ https
server webserver1 192.168.1.2 cookie webserver1 maxconn 5000
server webserver2 192.168.1.3 cookie webserver2 maxconn 5000
at 3:15 am
no mention on how to start stunnel service? The compiled stunnel does not install the startup script in /etc/init.d/ folder. How to start stunnel then?
at 12:01 am
How will this change given the later versions of stunnel have proxy option built into it.
xforwardedfor=yes is no longer supported.
at 2:42 am
Hi Bob,
Thanks a lot for this blog. These instructions were the foundation of my own haproxy SSL solution. The one thing that I thought was lacking was further SSL security between haproxy and the balanced servers. This may not be a problem for most people but I wanted to be extra vigilant. Rather than fighting with stunnel to get two instances running (one client and one server) I decided to use stunnel on the back end to secure the connection to the servers, and use pound on the front end.
Just in case you're interested I've documented it at http://www.philward.me.uk/?p=94
I have also added in some extra resilience in the form of a second standby load balancer and a system for monitoring the web servers in order to automatically adjust the haproxy configuration if one should suddenly fail.
Thanks again,
Philip Ward.
at 5:55 am
Also, if you want to use original address transparently in webserver:
Install apache module rpaf, which replaces REMOTE_HOST with value in X-FORWARDED-FOR.
If you want to bypass some redirects (e.g. from 443 to 80) for this specific server, also add
"reqadd X-Forwarded-From:\ " to https part.
Then in rewrite add:
RewriteCond %{HTTP:X-FORWARDED-FROM} !
Anyway this post was VERY useful for me, thanks.