Performance Research, Part 3: When the Cookie Crumbles

March 1, 2007 at 4:41 pm by Tenni Theurer | In Development, Performance |

This article, co-written by Patty Chi, is the third in a series of articles describing experiments conducted to learn more about optimizing web page performance (Part 1, Part 2). You may be wondering why you’re reading a performance article on the YUI Blog. It turns out that most of web page performance is affected by front-end engineering — that is, the user interface design and development.

HTTP cookies are used for a variety of reasons such as authentication and personalization. Information about cookies is exchanged in the HTTP headers between web servers and browsers. This article discusses the impact of cookies on the overall user response time.

HTTP Quick Review

Cookies originate from web servers when browsers request a page. Here is a sample HTTP header sent by the web server after a request for www.yahoo.com:

  HTTP/1.1 200 OK
  Content-Type: text/html; charset=utf-8
  Set-Cookie: C=abcde; path=/; domain=.yahoo.com

The header includes information about the response such as the protocol version, status code, and content-type. The Set-Cookie is also included in the response and in this example the name of the cookie is “C” and the value of the cookie is “abcde”. Note: The maximum size of a cookie is 5051 bytes in IE 6.0 and 4096 bytes in Firefox 1.5.

The browser saves the “C” cookie on the user’s computer and sends it back in future requests. The “domain=.yahoo.com” specifies that the browser should include the cookie in future requests within the .yahoo.com domain and all its sub-domains. For example, if the user then visits finance.yahoo.com, the browser includes the “C” cookie in the request. Since an Expires attribute is not included in this example, the cookie expires at the end of the session.

Here is a sample HTTP header for finance.yahoo.com sent by the browser:

  GET / HTTP/1.1
  Host: finance.yahoo.com
  User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; ...
  Cookie: C=abcde;

Notice that the “C” cookie, originating from www.yahoo.com, is also included in the request for finance.yahoo.com.

Impact of cookies on response time

The performance team at Yahoo! ran an experiment to measure the impact of retrieving a document with various cookie sizes. The experiment measured a static HTML document with no elements in the page. The primary variable in the experiment was the cookie size. We ran the experiment using a test harness that fetches a set of URLs repeatedly while measuring how long it takes to load the page on DSL. The results are shown in Table 1.

These results highlight the importance of keeping the size of cookies as low as possible to minimize the impact on the user’s response time. A 3000 byte cookie, or multiple cookies that total 3000 bytes, could add as much as an 80 ms delay for users on DSL bandwidth speeds. The delay is even worse for users on dial-up.

How big are users’ cookies set at the .yahoo.com domain?

Cookies set at the .yahoo.com domain affect the overall response time for users visiting web pages across the Yahoo! network. Figure 1 shows the percentages of Yahoo!’s total page views with various cookie sizes set at the .yahoo.com domain.

Figure 1. Percentage of Page Views with Various Cookie Sizes

About 80% of page views have fewer than 1000 bytes of cookies, which correlates to about a 5 to 15 ms delay for users on DSL bandwidth speeds. While the data shows that the majority of page views aren’t impacted by a significant delay, it also shows that about 2% of page views have over 1500 bytes of cookies set at the .yahoo.com domain. Although 2% sounds insignificant, at Yahoo! this translates to millions of page views per day, a compelling motivation for us to investigate this 2% and eliminate unnecessary cookies, reduce cookie sizes, and set cookies at more granualar domain levels.

In an earlier post about browser cache usage, one user made a comment about the side-effects of different browsers. Since Internet Explorer and Firefox have different implementations for the maximum size and number of cookies supported, we also analyzed the data by browser type and found no significant difference between the cookie sizes. It would be interesting to further investigate whether there is a difference in performance across browsers.

Analysis of Cookie Sizes across the Web

To show how Yahoo!’s cookie usage relates to those of other companies, we analyzed the cookies set by other popular web sites. For this experiment, we cleared all our cookies and visited only the home pages of these web sites. Table 2 shows between 60 and 500 bytes of cookie information included in the HTTP headers.

The data in Table 2 reflects only cookies set at the top domain levels to eliminate any cookies that may have been set by ads. The total cookie size for Yahoo! (122 bytes) in Table 2 differs from the cookie sizes indicated in Figure 4 because in this experiment we visited only the home pages of each web site. The data in Figure 4 reflect real users, many of whom visit multiple Yahoo! web pages. To illustrate, if tv.yahoo.com and movies.yahoo.com wanted to share information within a cookie, the cookie must be set at the .yahoo.com domain. The total cookie size set at the .yahoo.com domain for a user who visits multiple Yahoo! sub-domains is typically higher than the total cookie size set for a user who only visits www.yahoo.com.

Setting cookies at the appropriate path and domain is just as important as the size of the cookie, if not more. A cookie set at the .yahoo.com domain impacts the response time for every Yahoo! page in the .yahoo.com domain that a user visits.

Takeaways

Eliminate unnecessary cookies. Keep cookie sizes as low as possible to minimize the impact on the user response time. Be mindful of setting cookies at the appropriate domain level so other sub-domains are not affected. Set an Expires date appropriately. An earlier Expires date or none removes the cookie sooner, improving the user response time.

Share and extend: Bookmark with Yahoo! My Web | Bookmark with del.icio.us | digg it! | reddit!

21 Comments »

RSS feed for comments on this post. TrackBack URI

[…] Recently, the Ruby on Rails edge gained a new feature: cookie-based sessions. There’s pros and cons to this, but I’m curious to see how it plays out. It definitely reduces the shared components even further, but I wonder how performance will manage against other back-end stores, such as memcached. The YUI blog has a look at the impact of cookie size on performance that makes me think it might not be bad if you don’t store much of anything in it. […]

Pingback by Cookies, Rails and Performance at Pensieri di un lunatico minore — March 1, 2007 #

[…] The mad(in a good way) scientists at Yahoo! UI are at it again to learn more about optimizing web page performance. In part 3, they examine the impact of cookies on overall response time. The “takeaways” from this latest installment are: […]

Pingback by AJAX Matters Blog » Blog Archive » Performance Research, Part 3: When the Cookie Crumbles — March 1, 2007 #

So if tv.yahoo.com wants to set a cookie for movies.yahoo.com without affecting other Yahoo subdomains, should it pull an image or iframe page from movies.y.c with cookie info in the URL and let movies.y.c set the cookie itself? With appropriate XSS checks, presumably?

Comment by Bruce — March 1, 2007 #

It is possible to merge together multiple cookies into one by using
server side storage. It requires quick server side storage that is shared among all your web servers and domains. Not always practical, desirable or, depending on your software, even possible.

Another alternative is to use GET and POST arguments.

Are you planning on doing any research into the performance of GET and POST on web page speed? It would be very interesting.

Ciao!

Comment by The Doctor What — March 2, 2007 #

A related point: cookies also get sent when you load each image in a page. So if you have a lot of images, you may want to put them under a different domain than your cookies.

Comment by JW — March 2, 2007 #

Nice info Tenni I am going to be using Set-Cookie in the next few weeks so this was a nice background for me!

You might like Force Apache to output any HTTP Status Code with ErrorDocument

I setup an automated system to view all 57 Apache Response codes and ErrorDocuments, saving the headers and returned content for future reference. Use this page as a reference when designing scripts that use headers. Ex: 404 Not Found, 200 OK, 304 Not-Modified, 506 Service Temporarily Unavailable, etc.

When a Status code is encountered, Apache serves the header and the ErrorDocument for the error code. So if you can see any Header and ErrorDocument by causing that error on Apache.

For instance, if you request a file that doesnâ€™t exist, a 404 Not Found is issued and the corresponding ErrorDocument is served with the 404 Not Found Header. So we can see what Apache 404 Errors and Response Codes look like, but how do we cause errors for the 56 other Apache Response Codes?

Comment by DJ — March 4, 2007 #

[…] Performance Research, Part 3: When the Cookie Crumbles (tags: Cookies) […]

Pingback by All in a days work… — March 4, 2007 #

[…] Performance Research, Part 3: When the Cookie Crumbles Â» Yahoo! User Interface Blog (tags: Yahoo performance user-experience webperf) […]

Pingback by links for 2007-03-03 » dougmcclure.net — March 5, 2007 #

@The Doctor What: A fellow Yahoo, Iain Lamb, has done some awesome research comparing XHR GET and XHR POST for the new Yahoo! Mail product. In his research, he confirmed a “rather baffling finding: POST requests, made via the XMLHTTP object, send header and body data in separate tcp/ip packets [and therefore,] xmlhttp GET performs better when sending small amounts of data than an xmlhttp POST. If the amount of data you have to send to the server is small (less than 2k), I suggest you design your webservice / client application to use GET rather than POST (and that you try to keep the amount of YAHOO cookies sent on the small side, so that the request can go out in one packet).”

Comment by Tenni Theurer — March 5, 2007 #

cookieã®ã‚µã‚¤ã‚ºã¯ãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹ã«å½±éŸ¿ã‚’ä¸Žãˆã‚‹ã‹…

ç§ã¯ “å›žç·šãŒæ—©ããªã£ãŸç¾åœ¨ã€ãƒ‡ãƒ¼ã‚¿ã®è»¢é€ã‚ˆã‚Šã‚‚ã‚³ãƒã‚¯ã‚·ãƒ§ãƒ³ã®ç¢ºç«‹ã«ã‹ã‹ã‚‹ã‚ªãƒ¼ãƒãƒ¼ãƒ˜ãƒƒãƒ‰ã®æ–¹ãŒå¤§ãã„” ã¨ã„ã†æ ¹æ‹ ã®ãªã„ä¿¡å¿µã‚’æŒ……

Trackback by bits and bytes — March 6, 2007 #

[…] Javascript dosyalarÄ±mÄ±z ve kullandÄ±ÄŸÄ±mÄ±z yÃ¶ntemleri irdeleyerek web sitelerinin performansÄ±nÄ± arttÄ±rmaya yÃ¶nelik makale dizsinin Ã¼Ã§Ã¼ncÃ¼sÃ¼ Ã§erezleri kontrol altÄ±na almak Ã¼zerine. Link […]

Pingback by Fatih HayrioÄŸlu’nun not defteri » 7 Mart 2007 Web’den seÃ§me haberler — March 7, 2007 #

[…] Â» Yahoo! User Interface Blog Posted in performance, optimization by kuma on the March 5th, 2007 Performance Research, Part 3: When the Cookie Crumbles Â» Yahoo! User Interface Blog500byteã§15msã£ã¦å›žç·šé…ã™ãŽã€‚250kbpã®ãƒ¬ãƒ™ãƒ«ã€‚æ—¥æœ¬ã§ã¯ã»ã‹ã®ã¨ã“ã‚ã§å¯¾ç–ã—ãŸæ–¹ãŒã‚ˆã•ãã†ã€‚ […]

Pingback by ZeroMemory Bookmarks » Performance Research, Part 3: When the Cookie Crumbles Â» Yahoo! User Interface Blog — March 27, 2007 #

[…] http://yuiblog.com/blog/2007/03/01/performance-research-part-3/ […]

Pingback by Top methods for faster, speedier web sites — May 23, 2007 #

[…] fewer HTTP requests: This also affects cookies. Eliminate unneccesary cookies, keep them small, set them at granular domain levels (e.g. […]

Pingback by Learning the World » Website Performance Tweaks, Part Two — June 24, 2007 #

Controlling the size of your cookie and the domains/subdomains you use to share them can quickly become vital, espacially if you have to deal with a http/https path.

One lesson I learned after managing the QoS of a telco web site for several months is that https uses really less connections to connect to the server, and that securing requests takes much longer. If you a have a big cookie (such as and identity cookie, ususally 1024 bytes long), you find yourself creating secure requests for each file you need to download, sending that big cookie over a DSL line (which is usually smaller in bandwidth from the browser to the server), and doing so an inappropriate number of times (because all your cache is down the trash, with https).

The solution? You need to accomodate : use an alternate domain or subdomain or server to serve static files (without cookies), with a cname to have two parallel downloads and good cache directives.

Use frames, if you have a heavy graphic design, and secure only the parts that need it.

Try to keep the number of css and js files to a minimum : each file you download costs you, even before you start getting the response, and the browser WILL NOT RENDER the page unless it has downloaded all css it needs.

I recommended all these actions to the different engineering teams that are in charge of the self-care, store and portal of the company. We saw wonderful successes, withe a 50% decerase in homepage load time, or a 10% decrease of bandwidth use while we had a 20% increase of traffic (page views). It works!

Comment by Cyril — June 25, 2007 #

[…] Performance Research, Part 3: When the Cookie Crumbles […]

Pingback by Mrasnika’s Lair » EÑ…, Ð¼Ð°Ð»ÐºÐ¾ Ð²Ñ€ÐµÐ¼Ðµ Ð·Ð° Ñ‡ÐµÑ‚ÐµÐ½Ðµ Ð½Ð° Ð±Ð»Ð¾Ð³Ð¾Ð²Ðµ — August 1, 2007 #

[…] When the Cookie Crumbles […]

Pingback by Client side web site performance â€” onenaught.com — August 6, 2007 #

[…] experiments conducted to learn more about optimizing web page performance (Part 1, Part 2, Part 3, Part 4). You may be wondering why you’re reading a performance article on the YUI Blog. It […]

Pingback by Performance Research, Part 5: iPhone Cacheability - Making it Stick » Yahoo! User Interface Blog — February 6, 2008 #

Reducing cookie size is only one solution to the problem of cookie-related latency.

Another is setting the cookie path attribute differently from the ubiquitous “/” means that the cookie will be resent upstream by the browser for each external file requested, including images, javascript and CSS files that have no use for cookies. Setting your static content in a different folder than your code and NOT specifying path=”/” solves this problem.

Yet another method is setting the cookie’s domain attribute: set your static content to a different subdomain than your code and limit your cookie’s domain to the code’s domain.

Comment by Yair — March 19, 2008 #

[…] five great articles that covers reducing the number of HTTP requests, explains web browser caching, the effect of HTTP cookies (co-written by Patty Chi), parallel downloads (co-written by Steve Souders) and details on caching […]

Pingback by More Optimization for your site — March 29, 2008 #

[…] more information check “When the Cookie Crumbles” by Tenni Theurer and Patty Chi. The take-home of this […]

Pingback by To Speeding up Your Web Site « Sanantoniowebdesign Weblog — August 7, 2008 #

Yahoo! User Interface Blog