How better Caching helps Frankfurt’s Airport Website to handle additional load caused by the Volcano
Along with so many others I am stranded in Europe waiting for my flight back to the United States right now. The Volcano not only impacts flights across Europe but also impacts web sites of airports, airlines and travel agencies around the world. Checking my flight status on Sunday was almost impossible. The website of Germans largest airport – Frankfurt am Main – was hardly reachable. No wonder as I assume that their page just got hammered by thousands of additional page requests of frustrated travellers. Now it’s Tuesday and the website is back to “almost acceptable” response times. Time for me to analyze the current web site as I’ve done with others such as vancouver2010, utah.travel.com or masters.com.
Status Quo: Too many resources and wrong cache settings
Looks good – but wait. Let’s not be deceived. We still see a very high value on Server Transfer time. Based on my experience this means that – even though content is retrieved from the cache – the browser sent HTTP requests to the server for each individual resource to “ask” whether the content has been modified (IF-MODIFIED-SINCE) since the last time the resource was downloaded. This is OK if I haven’t checked the web-site for weeks or months, but it is not ok if the last page visit has only been minutes ago.
A closer look at the Network Requests view reveals the problem. The Expires Header is actually set “to the past”. I recorded my session on April 20th 2010 at 09:38GMT. The Expires header is set to April 19th – that was yesterday. That is the reason why my browser has to send an HTTP Request to the server for every “cached” element to check if there is a newer version of the resource on the server or not. The Server Column shows us how much time is spent for each request on the server to determine whether the resource has changed or not. The Wait column tells us how long individual requests had to wait to be processed (this is again caused by the physical network connection limitation – only 2 physical connections are available for a domain – all other requests have to wait).
The Network view shows us almost all HTTP Headers. Due to the nature of the dynaTrace AJAX Plugin in IE we do not get ALL HTTP Headers but we get the most interesting ones. Our users have already requested this feature on our Community Wish List. Right now I propose to use a Network Sniffer or Proxy such as MS Fiddler, HTTP Watch, Charles, … in case you need more detail than the AJAX Edition provides.
How to improve the performance
Theoretically it is pretty simple to improve performance on sites like this. I say theoretically because some of the proposed changes require some work and changes on the web server or web deployment. Here is a list of proposed changes and an estimated performance gain:
- Use HTTP 1.1 or at least Connection: Keep-Alive: The web-server runs on HTTP 1.0 and forces the browser to close the physical connection after each request. Use Connection Keep-Alive to avoid unnecessary reconnect efforts.
- Estimated Gain: 100-200ms (check the Connect Column in the Network View)
- Far Future Expires Header: for those elements that change very infrequently use an Expires Header in the Far Future
- Estimated Gain for returning users: 4-6s (depending on how many objects can really be cached long time)
- Merge CSS: Merging all 22 CSS files into a single CSS file would eliminate Wait Time and reduce Server and Transfer Time due to reduced HTTP Roundtrips
- Estimated Gain: 1.3s in Wait Time, 1-2s in Server-Time and Transfer Time (assuming we can merge them)
- Estimated Gain: 300-500ms
- Domain Sharding: Spreading the 75 images served from the main page on 2 additional image sub-domains allows the browser to download 4 images in parallel. It also allows other content from the main page, e.g.: AJAX Requests, … to be downloaded without waiting for image downloads
- Estimated Gain: 2-3s
Small things that are often missed – liked wrong Expires header – make a huge difference in web site performance. If the website of Frankfurt’s Airport would have followed some of the best practices from Google or Yahoo or those that we give here at our dynaTrace Blog I am pretty sure many travellers would have been able to reach their web-site on Sunday (even though we would have still been stranded).