Improved HTTPS Performance with Early SSL Termination

From: http://blog.filepicker.io/post/29422604907/improved-https-performance-with-early-ssl-termination 

In the continuous quest for a better user experience, performance plays a large part. Therefore, at Filepicker.io, we go to great lengths to minimize latency. In this post, I’ll go into one example of how I did so recently.

Filepicker.io ensures your privacy by sending all our traffic over HTTPS. However, since our web servers are located on east coast, HTTPS results in a measured performance penalty of over 200ms on every cross country connection since it requires two additional round trips due to handshakes. Wanting to provide the best experience possible, we’ve engineered early SSL termination to mask this latency.

Anatomy of an HTTPS Connection

First, let’s see where the additional latency of HTTPS comes from. HTTPS works by tunneling HTTP over an encrypted TLS connection. The initial handshake looks like so:

Client: TCP SYN
Server: TCP SYN+ACK
Client: TCP ACK
———begin TLS———-
           TLS Hello
Server: TLS Hello
           Server Key
Client: TLS Ack
           Server Key check
           Client Key Exchange
Server: TLS Handshake
———-Begin HTTP———
Client: HTTP Request (GET)
Server: HTTP Response

As you can see from the https connection initiation, HTTPS requires four round trips to make a single HTTP request while unencrypted HTTP only requires two. This means that a browser on the west coast will take upwards 4 * 96ms (cross country round trip time) = 384 ms (more like 584ms due to server certificate checking) to make a single HTTPS request in comparison to 2*96ms = 192ms for an unencrypted HTTP request.

So how do we get around this? I engineered a solution that masks most of the latency of HTTPS that we call early SSL termination.

The ideal solution would be to run our web servers all around the world, as close as possible to the consumer. However, this is not possible without a great deal of engineering effort since the masters of our replicated database have to be contained in one region for performance reasons.

Early SSL Termination

Instead of distributing our web servers, we did the next best thing: distributing where we terminated our SSL connections. By cutting down all the round trip times except the HTTP request, we can theoretically save 3*96ms = 288ms of latency.

The basic premise of how this works is that we set up elastic load balancers and instances in multiple EC2 regions and terminate SSL there, close to the consumer. Nginx then processes the http requests and forwards them to a pool of warm keep-alive HTTPS connections to the main web servers so that only the actual HTTP request is sent, thus ensuring each HTTPS request only has to do one cross country round trip.

For example, if my client is on the west coast and my web servers on the east coast, my HTTPS request would connect to an instance in the Northern California Region of EC2 and do the SSL handshake there performing the 3 required round trips in less than 3*15ms (west coast client to west coast instance time) = 45ms. The actual HTTP request would then be sent to the intermediate instance which then forwards it on over an already established HTTPS connection and gets a response back in 1 round trip from the main application servers, then sending it back to the consumer in a final round trip time of 15ms + 86ms = 101ms.

Overall, at filepicker.io, for a single HTTPS request, early SSL termination cuts the time from 384ms to 146ms, over 200ms improvement. In Europe and Asia, the improvement is even more dramatic.

Discussion on Hacker News