About the Author

David Jones is an APM Technical Evangelist and has been with Compuware for 8 years, he has close to 20 years’ experience working with web technologies at organizations such as S1 Corp (Atlanta), Broadvision (Bay Area), Interleaf/Texcel (Waltham), i4i (Toronto) and SoftQuad (Toronto).

2014 Super Bowl Tips to Avoid Ad Site Fails

Last Sunday the Seattle Seahawks dealt Denver one of the worst beatings in recent Super Bowl history; however, the only highlights of the broadcast were the commercials. They ranged from serious and thought-provoking to funny and quirky.  Each ad was meant to do one thing:  drive eyes to a brand.  With most of the population watching with their phones and tablets, every advertiser’s site had to be ready for those eyeballs.

Everyone wants to interview the winners and  losers after the game.  There is a dissection of every drive as analysts want to understand key aspects of success and failure.:

  1. MVPs and who’s to blame
  2. The Breakdown on both sides
  3. What to do for next season

I, of course, love football, but I also love watching Super Bowl ads and how they perform.  I love looking at who was the fastest, who was the slowest, and understand why.  The Internet is a level playing field on which everyone (with enough money) has the same options as everyone else; o when it comes to game time strategy, why do  sites perform so differently?

MVPs and Who’s To Blame

Here is our wrap up of how the ad websites aired during the course of the game:

Ad Test Name

Average Response
Time (sec)

Success
(percent)

HM – SB Ad – Backbone

0.46

100.00

Carmax.com SB Ad – Backbone

0.50

100.00

Godaddy SB Ad – Backbone

0.63

100.00

Bank of America SB Ad – Backbone

0.68

100.00

WeatherTech.com SB Ad – Backbone

0.87

100.00

Maserati SB Ad – Backbone

0.93

100.00

Butterfinger Nestle SB Ad – Backbone

1.10

100.00

Sonos SB Ad – Backbone

1.15

100.00

Old Spice SB Ad – Backbone

1.19

100.00

Honda SB Ad – Backbone

1.48

100.00

Microsoft SB Ad – Backbone

1.48

100.00

Audi SB Ad – Backbone

1.50

100.00

Cure.com SB Ad – Backbone

1.51

100.00

Honda Accord SB Ad – Backbone

1.54

100.00

Wonderful Pistachios SB Ad – Backbone

1.55

100.00

Chrysler – SB Ad – Backbone

1.56

100.00

Budweiser SB Ad – Backbone

1.82

100.00

Subway SB Ad – Backbone

1.86

100.00

Paramount SB Ad – Backbone

1.87

100.00

Axe Unilever SB Ad – Backbone

1.96

100.00

Cherokee SB Ad – Backbone

1.97

100.00

Coca-Cola – SB Ad – Backbone

2.05

100.00

Sprint SB Ad – Backbone

2.05

100.00

Squarespace SB Ad – Backbone

2.40

100.00

Alex and Ani SB Ad  – Backbone

2.41

100.00

Cheerios SB Ad – Backbone

2.49

100.00

Beatsmusic SB Ad – Backbone

2.59

100.00

Heinz SB Ad – Backbone

2.93

100.00

GoPro SB Ad – Backbone

3.15

100.00

Jaguar SB Ad – Backbone

3.28

100.00

Jack In The Box – SB Ad – Backbone

3.40

100.00

Radio Shack SB Ad – Backbone

3.42

100.00

TurboTax SB Ad – Backbone

3.45

100.00

Chevrolet SB Ad – Backbone

3.53

100.00

Budweiser SB Ad – Backbone

3.62

100.00

Chobani SB Ad – Backbone

3.63

100.00

Hyundai SB Ad – Backbone

3.91

100.00

Oikos SB Ad – Backbone

3.94

97.61

Mars SB Ad – Backbone

4.10

100.00

Volkswagen SB Ad  – Backbone

4.19

100.00

T-Mobile – SB Ad – Backbone

4.47

100.00

Intuit.com SB Ad – Backbone

4.93

100.00

Soda Stream SB Ad – Backbone

5.14

84.62

Bud Light SB Ad – Backbone

5.49

100.00

DraftDayTheMovie.com – SB Ad – Backbone

5.61

100.00

Noah Movie – SB Ad – Backbone

6.22

95.17

Toyota SB Ad – Backbone

6.78

99.52

Time Warner Cable SB Ad – Backbone

7.06

100.00

Pepsi SB Ad – Backbone

7.15

100.00

Doritos SB Ad – Backbone

7.88

100.00

Kia SB Ad  – Backbone

8.37

84.47

 

Tracking the ads for the Super Bowl can be tough as some advertisers don’t indicate whether or not they are advertising during the Super Bowl; others promote their ads well in advance.  To compensate, our team added tests during the game as the ads aired but the methodology we used was the same for all.

We tested the ad URL’s using real browser agents from end-user locations across the US. The tests ran from the following locations every 10 minutes during the game:

  • CA: Los Angeles – Verizon
  • CA: San Jose – AT&T
  • FL: Miami – Internap
  • IL: Chicago – Level3
  • MO: St. Louis – SAVVIS
  • NY: New York – Sprint
  • TX: Dallas – AT&T
  • VA: Reston – Savvis
  • WA: Seattle – Internap

We call this methodology a “9” Box as it divides the US into East, Mid-West and West; with three locations in each area running north to south.  This gives us good coverage across the continental US; we recommend this approach for basic synthetic monitoring.

The browser agents doing the tests are the same as a real user opening a browser and making a connection to the page.  It performs actions such as: resolving the DNS address(s) for the Ad as well as  the Ad’s content including third parties); establishing the TCP connection(s) to all the domains contributing to the page; downloading the base ad page and reads the HTML, executing all the Javascript and CSS;  downloading all the images and content being requested by the HTML and Javascript;   calculateign how much time it takes the server to respond to request (First Byte Time) and then how much time it takes to download all of the content requested by the page. This allows us to understand which company had the fastest response time, which had the slowest, and how each got that way.

The Breakdown On Both Sides

Here are the Top 10 ads ranked by speed and performance.  With these results we are looking for an understanding of how much content was being delivered (Average Number of Bytes), the number of 3rd parties being used (Average Number of Hosts), and the complexity of the page being delivered (Average Number of Objects).

Top 10 Ranked Ad Web Sites during Super Bowl 2014

Top 10 Ranked Ad Web Sites during Super Bowl 2014

Not so fast, Maserati! While Maserati may have been in the Top 10 in terms of performance, we know that their site failed.  In order to do proper performance testing one needs to track the page performance and to validate whether or not content is being deliverly properly.  When compared to the ads from other car companies it is pretty clear that the Maserati page was missing promised content (check the link above for a screen shot). It wasn’t until they fixed the issue that the page performed better than any other car ad.

In comparison here are the slowest 10 Ad pages.

Slowest 10 Ad Web Sites during Super Bowl 2014

Slowest 10 Ad Web Sites during Super Bowl 2014

Kia, Pepsi, the movie trailer ads (Draft Day, Noah) all suffered from trying to deliver too much content;  other sites suffered from having too many 3rd parties (number of Hosts) or being too complex (having too many object calls

What To Do For Next Season

Our impression of the Super Bowl advertisers is similar to our impression of the holiday season retailers. They need to:

Get Back To Basics!

  1. Reduce the amount of content you are trying to deliver.  If you look at some of the common performance indicators of the Top 10 ads, you can see they kept their content optimized in terms of size (page weight).  The smaller the web page, in terms of byte count, the faster it can be delivered.  This has additional benefits in terms of keeping costs down as most ISPs and CDNs charge based upon how much content is being delivered.
  2. Reduce your third parties and simplify your pages.  If you look at some of the key performance indicators for the Top 10 ads you will see that they tend to use less 3rd parties (i.e cutting down on the number of analytics/tracking tools and the number of Social media calls). When you are trying to deliver an ad to as many people as possible, question whether or not that the right time to also call your social media content. Simplification also means keeping your Web 2.0 (AJAX) functionality in check.  Just because you can do something neat with JavaScript doesn’t mean that you should.
  3. Above all:  TEST! Do Super Bowl teams win without practicing plays?  Advertisers need to test their ads from an end-user point-of-view prior to going live.  All of the slowest advertisers could have easily made some simple design changes that would have dramatically sped up their performance.

What Next?

The Super Bowl is a unique situation for organizations where Marketing depends on IT to make sure that the lights stay on when the rush comes to the front door.  You can read more about how to ensure this does not happen to you here or here.  If you think you might have a problem you can test your site today here for free. Stay tuned to the blog as we take a closer look at why some of these sites succeeded and some of the failed and what you can do to avoid making the mistakes.

Comments

  1. Great breakdown! One thing to note is the columns in the MVP table are misaligned, it’s not immediately clear if I’m scrolling down which entry belongs to which row

Comments

*


three − 2 =