Why Your A/B Testing Program isn't Working

By Kurtis Morrison, VP Client Services at EyeQuant


Over the past 4 years I’ve met somewhere between 500 and 1000 conversion optimization practitioners. I meet more every week, and with every person I meet I try and learn a little something. I ask lots of questions. Lately, one of my favourite questions is this: What percentage of your A/B tests are “winners” (i.e. they produce a statistically significant uplift in conversion)?

It seems simple enough. After all, the ultimate KPI for any conversion optimization program is uplift. Without uplift, there is no measurable ROI of conversion optimization at all. There’s no tangible reason for management to take it seriously as a function. So you’d think that people in CRO - who spend all day looking at metrics and data - would know their own numbers, right? Yet in most cases, the people I talk to have only a rough idea of what their win-rate is, and many don’t really know. 

(FYI: the reported "win rates" range from 20% to around 70%)

I also ask about the cost of testing. Most people think about the cost of testing in terms of the wages for people involved in CRO, and the price of the tools they use.

I rarely hear about the less tangible factors like opportunity cost, but for any online business, there’s only enough traffic for X number of tests per month. Every testing slot you use = one that can’t be used on other test ideas that might potentially have a bigger impact. And every time you run a test where 1 (or more) variants underperforms the control, you lose real revenue. It’s called “testing”, but there are real customer experiences and cash at stake. These less obvious costs can be huge.

By comparison, whatever price you’re paying for your Optimizely (or other) subscription is fairly insignificant. And yet it’s amazing how many people would happily run an A/B test on a hunch rather than pay a few bucks for some user testing (for example) to validate their idea before they commit real traffic and revenue to it. That’s why I think CRO has grown steadily over the past 5 years, but never really saw “hockey stick” growth as a discipline. In most cases, testing programs aren’t run like a “business”, i.e. a deliberate process with carefully considered benefits and costs.

But what if they were? I can think of 3 major ways that teams might change in terms of the way they work:

  1. Optimizers would look for multiple sources of data that tell the same story about their website. An insight from a user test, a survey, or a heatmap on its own wouldn’t be enough because teams would recognize that there’s too much at stake. Teams would look for multiple signs pointing the same direction.

  2. The amount of attention paid to the quality of test variants and the process of designing test variants would dramatically increase. Today, teams spend most of their time and effort on defining what’s wrong with their website, while the actual solutions part is rather unsophisticated. If testing were treated as serious business, there would be careful checks in place to ensure that the variations are true to the original hypothesis, and some validation work would be done to ensure those variations have a decent chance of winning.

  3. Teams would focus more on test velocity. After all, additional revenue from testing simply comes down to your average revenue uplift multiplied by the number of tests. If teams took more time to estimate the financial impact of increasing test velocity, I think we’d see a lot of companies investing in more agile processes and technical infrastructure.

If more teams were doing these things, I think conversion optimization would definitely gain a lot more traction in the C-suite, and we’d see a huge increase in resources dedicated to testing. And wouldn’t that benefit all of us?

If you have an opinion on this, let me know in the comments or feel free to get in touch with me at kurtis@eyequant.com!


Cyber Monday: Rating the Best and the Worst Mobile Experiences with A.I.

It’s Tuesday morning and retailers across the world are breathing a collective sigh of relief. Although the holiday shopping season still has nearly 4 weeks to go, the high-stakes, ultra-stressful Black Friday and Cyber Monday are behind us. For retailers, the last few days were all about executing on the strategies that have been carefully planned for months. According to a piece in the Washington Post, a common goal this year was to win over a coveted new breed of omnichannel shoppers who “ping-pong from physical stores to laptops to smartphones, and a purchase can come via any of these avenues at any time.”

For most retailers, mobile is their “problem” channel – the one they haven’t perfected yet. That’s bad news, because today’s shoppers rely heavily on their mobile devices. During the holidays last year, approximately one third of all eCommerce purchases were made on a smartphone (source), due to a major spike in mobile traffic – a trend that seems to have continued throughout 2014.

To make matters worse, shoppers are especially fickle when they’re on the mobile device. According to Google, 40% of mobile customers have turned to a competitor’s site after a bad mobile experience. That’s especially harsh considering that a decent chunk of mobile traffic comes from people who are physically in a store already.

So which of the top retailers seem to have pulled it off?

Today, we’ve looked amongst the biggest online retailers in the US to find the best and worst mobile designs – as rated objectively by the EyeQuant A.I.

EyeQuant is trained with large amounts of data from user studies and eye-tracking research to provide instant, objective insight into how real people are likely to react to certain designs. Screenshots of the designs were captured during Cyber Monday.

The analysis results you'll see below are combinations of the EyeQuant Perception Map and Clarity Score. The perception map shows which content will be most eye-catching in the first 3 seconds of a user landing on the page (the illuminated areas), while the Clarity Score shows how users would rate the design on a 0-100 scale (0 being extremely cluttered, 100 being essentially a blank page).

The Best



The struggling Chicago-based retailer has made major investments in eCommerce user experience over the past couple of years, and it’s really starting to show. When shoppers arrived on the mobile site on Cyber Monday, they immediately saw a banner prompting them to “view Cyber Monday deals” with no distractions (see attention map). The page is extremely “clean” according to the Visual Clarity ratings, putting up an impressive score of 91. Bonus points for an extremely accessible search bar.






Like Sears, the Japanese electronics giant has had a tough time lately with brick-and-mortar stores. 20 of 31 US retail locations have been closed - or will be closed - before January 1st. The official reasoning is to streamline costs and focus on existing partner relationships with other retailers. Online, however, Sony looks poised for great results this year - at least judging by the mobile experience. The first thing users see is an announcement that Cyber Monday deals are here, and the design puts up an incredible Clarity Score of 92.





The Framingham, MA giant - which traditionally has a cluttered and distraction-filled website, turned in an impressive mobile performance on Cyber Monday. The eye flow on this one is fantastic. Users see the announcement that Black Friday deals are here. Then they’re guided to a specific offer that rocks, and then users are given the opportunity to view more deals. Meanwhile, the clarity score is a respectable 78.

Like Sears, the Japanese electronics giant has had a tough time lately with brick-and-mortar stores. 20 of 31 US retail locations have been closed - or will be closed - before January 1st. The official reasoning is to streamline costs and focus on existing partner relationships with other retailers. Online, however, Sony looks poised for great results this year - at least judging by the mobile experience. The first thing users see is an announcement that Cyber Monday deals are here, and the design puts up an incredible Clarity Score of 92.

The Not-so-great:


There is no excuse for a top retailer not to have either a responsive website or to redirect users to a dedicated mobile site. But Macy’s, the retail giant founded in 1858, sends users to a scaled-down version of the desktop website. Not only does that make it harder to use, but the first impression suffers a lot. The big problem is the miserable clarity score of 31 - which was the worst score we saw amongst the major retailers. Meanwhile, user attention is initially dominated by 2 competing “sale” messages - both telling you to “hurry up”. Ironically, the part of the page that actually tells you WHY you should hurry up (the 15% discount) doesn’t really stand out that much because of a mediocre colour choice (black on red).


Victoria’s Secret

People coming to your mobile site on Cyber Monday are looking for deals, and they are moving fast. But what does the Victoria’s Secret site do? Immediately direct user attention towards a hero shot that conveys very little. Meanwhile - despite multiple sale offers -the words Cyber Monday aren’t used anywhere, which leaves the user wondering: “are these deals any better today than a normal day”? To top it off, the page puts up a weak clarity score of 48.







Williams-Sonoma suffers from a mediocre clarity score (51), but the biggest problem is that users arrive on the page and are immediately presented with what looks like an enterprise pricing table rather than a great deal on home furniture. The call to action isn’t visible, either.







Good News

The good news is: no matter what happened this year, there's always room for improvement next year! What do you think of these ratings? Let us know in the comments! Want to try out EyeQuant for yourself? Claim 2 free test credits here.

Why The New Google Search Ads Design Is a Subtle Work of Genius

It's official - Google has rolled out a major redesign of their search results and search ads. The company line, as outlined by Jon Wiley - Google's lead designer for search - is that the new design improves "readability and creates an overall cleaner look", while the redesign of the ads is "making the multi-device experience more consistent." Google's desktop ads now do match the design of their mobile versions and achieving multi-device consistency certainly is a great reason. We'd like to take a data-informed guess on what other good reasons Google might have had for this major revamp of their most important interface. 

Readers of this blog will know that our mission is to teach computers to see designs like humans do - using neuroscience and machine learning. In this article we'll make use of our EyeQuant technology to better understand how Google's new ad design affects viewers, and thereby, clicks.

First things first, some of our assumptions for the analysis:

1. It's unlikely for Google to roll out a relatively major design change without having tested its effect on AdWords CTR (i. e. Google's main source of revenue) first. It's pretty safe to assume that the new design performs at least as well (and most likely better) in terms of CTR than the old one.

2. Google is still commited to its 'Don't Be Evil' motto, which in particular affects any conflicts of interest in serving their two most important stakeholders: search users and advertising customers.

Now, as the headline gives it away, we do believe that the redesign is a subtle work of design genius. Here's why.

Google's new Ads now very much 'blend in' with the organic results, all while attracting MORE attention, and all without being clearly evil. 

Yep, Google somehow squared the circle here. When designing ads, one is usually torn between:

  • aligning the ad's design to the content in order to battle banner blindness and drive, ahem, involuntary clicks
  • making the ads stick out visually to attract involuntary eye-movements (the most extreme measure being the use of motion).

The first strategy usually implies a sacrifice of direct ad visibility and its 'pop out' effect, and worse, moral principles.

Yet, Google's new ad design manages to achieve the following (besides its officially announced goals):

1. It does clearly label the ads as such - one could even argue that the labeling is clearer than before, as now every ad is explicitly labeled. Not evil. 

2. At the same time it makes the ads blend in much more with the organic results. Ethically, this is a bit of a tricky move, in particular since they had banned a similar type of design explicitly from being used by AdSense publishers. But hey - every ad is labelled as such. 

google search ads redesign

3. The best part? The new layout attracts MORE design-driven attention to the ads than the old one did. 

We ran EyeQuant tests on both the old (~Q1 2013) and new SERP layout to see in which version the upper ad unit would generate more attention - solely based on its design and common viewing behaviours and patterns. EyeQuant results are 90% equivalent of what an empirical study with >25 subjects would provide. The following heatmaps of both the old and new ad units design show the respective probability of an area to be fixated in the first few seconds of exposure - the top 3 ads are marked up in pink:

eyequant eye-tracking google redesign

As it stands, the results show that the new design generates significantly more attention for the top 3 ads than before - especially for the desirable (and expensive) #1 ad. The attentional pull from the organic search results remains unchanged.

It probably goes without saying WHY all of this is good for Google.

But HOW did they do it? There are several factors involved, and we can reverse-engineer them with a bit of knowledge on how human visual attention works:

  • They increased the font size, which typically drives more attention (but not always and only to a certain extent!)
  • The luminance contrast between text and white background is now higher (blue on white) than it used to be in the old version (blue on AdWords skin tone) - luminance contrast is one of the most basic and most powerful drivers of visual attention.
  • The yellow "Ad" logo provides additional color contrast without being overtly aggressive (low luminance contrast between yellow and white), and it does so in every single ad! 

In conclusion, this is a supremely clever, subtle redesign and there's lots of reasons to believe that it will effectively drive eye-balls, clicks and ad revenue. 

Full disclosure: While several Google teams are using EyeQuant as customers, EyeQuant was not involved in any part of the redesign process.

And here is our own ad: you can try EyeQuant for free on your own designs - all it takes is an email, a URL and 7 seconds of your time. Test it out now!

If you'd like to get a more detailed look into how you can use design to influence visual attention in a systematic and predictable way, check out our quick 8 minute introduction right here:

The 3 Most Surprising Insights From a 200 Website Eye-Tracking Study

eye-tracking website mythsAt EyeQuant, we do a lot of eye-tracking as part of our mission to teach computers to see the web like humans do. The main purpose of our studies is to find the statistical patterns that power our attention models (which you can use to instantly test your websites!) Today, we're sharing 3 of the most surprising insights we found. 

A lot of you have asked us about general rules of thumb around what drives (and doesn't drive) attention - in this post you'll learn why rules of thumb are difficult to establish and how a lot of the common ideas we have about human attention are more complicated than they seem. In fact, what you're about to read is going to be rather surprising and we're hoping to dispel some common myths about attention and web design with data. :)

METHOD: We're looking at data from one of our recent eye-tracking studies with 46 subjects who were purchasing products on 200 AdWords eCommerce pages. We recorded 261,150 fixations in total and users we looking at each webpage for 15 sec (+/- 6 sec) on average. The study was conducted in the Neurobiopsychology Lab at the University of Osnabrueck, Germany.

DISCLAIMER: Since the purpose of this study was to further expand EyeQuant's predictive capacities, we're also providing EyeQuant's results for comparison next to the empirical data - please note that these predictions are based on a new EyeQuant model that's currently in early testing, but are already quite close to the real thing (currently this model provides over 75% predictive accuracy (AUC, warning: math), whereas our standard model achieves over 90%).

Myth #1: "Faces always & instantly draw attention."

This is probably one of the most universal design assumptions about human attention you'll find on the internet: "as humans, we're naturally wired to always seek out and look at at any available faces first."

Roughly correct - except for when it isn't. The truth is that as humans we do really like faces. We'll look at them sometimes. We probably even have a dedicated brain area involved in processing facesHoweverwe look at them much less often than you would typically believe. 

The data (click images to open a large version in a new tab):



Not convinced? Below you'll find a lot more examples - from beautifully designed eCommerce shop to web 1.0 wall-of-text. We're not saying faces don't attract attention at all and are never looked at. Our data just shows that faces aren't the powerful attention-grabbers as one usually thinks they are. 

eye-tracking faces What about guiding user attention through faces? 

This is another popular assumption which seems to make a lot of sense: we're social beings and user gaze follows the gaze of faces on a website. Again, that's true, except for when it isn't:

eyequant vs eye-tracking validation

What's going on here? Our careful, explorative hypothesis is this: looking at a face does provide a sort of emotional buzz, so we may remember looking at them more than we do remember looking at other things. This might lead to wrong conclusions about general viewing behaviour.

Watercooler conclusion: "Faces are emotionally powerful, but they don't always attract as much attention as we think they do."

Myth #2: Large text instantly draws a lot of attention.

"Large text is a great way to attract user attention" is another rather popular idea about how attention works online. However, our data shows that it usually doesn't work. In a lot of cases big fonts even seem to have a negative effect on attracting attention:

Screen Shot 2014-01-14 at 19.23.52

eyequant eye-tracking

What's going on here? Our careful, explorative hypothesis is this: there may be an element of "banner blindness" involved. At the same, extremely large letters might be less readable for the human eye as well.

Watercooler conclusion: "Big typography is visually loud, but not at all a safe way to grab user attention. We need to look into other ways as well."

Myth #3: "The magical word 'FREE' always pops out."

It's true: economically, nothing beats 'FREE'. But does this also mean that the word pops out to users immediately when they're visitig a page? Our data says otherwise.

eyequant vs eye-tracking

EyeQuant validation eye-tracking

Watercooler conclusion: "'Free is a powerful semantical tool. We shouldn't rely on it as our main attention grabber though!"

Conclusions: don't rely on rules of thumb. Testing always beats guessing.

Rules of thumb are fun. They're simple. And the more complex the thing is they're trying to explain the more appealing they become. Alas, that's also where they often fail - and visual attention is a rather complex, extremely context-driven system that cannot be captured in a set of simple rules.

What we're doing at EyeQuant is to combine large amounts of data like the study above in lightning-fast computer models. As you've seen, our predictions come close to what you'd get from a real study, so if you're curious to get results for your own website, just test it for free in our web app.

If you found this article interesting, you should talk to us on Twitter!


When You Want to Convert, Less is More

When I'm chatting with clients, partners, and prospects, I find myself talking a lot about the idea of “visual clutter”. My colleague Bitsy has written extensively on the topic of “attention-driven design”, but I wanted to add my take on the issue.

This morning I had a call with an EyeQuant user who's starting a new conversion optimization agency in the UK. In his past life he was a SEO expert, so today he was analyzing some of his SEO clients' websites with EyeQuant to spot some opportunities for improvement.

When we looked at the Perception Map for one particular website, the 3 most important pieces of content were seen right away. At first glance, EyeQuant was suggesting this was a pretty good page.

Not so fast, I said. Looking at the Attention map, we could see that there was an orange-yellow haze over many parts of the page. Visually, user attention was scattered across the page. The page was simply too busy.

Perception and Attention Map Examples

“Visual Clutter” annoys users. Time and time again, we see that focused pages outperform crowded screens.

Here's my favourite (extreme) example. No disrespect to the folks at Cruise, but this is what their landing page looks like:

Cruise Homepage

You're probably thinking “wow that's busy”. EyeQuant agrees. Check out the Attention Map for this same page:

Cruise Attention Map

There is so much content competing for attention on this page, that users are almost surely overwhelmed.

And yet, when a website isn't converting, many people rush add content. More benefits listed. More special offers. More options. More testimonials. It's a natural reaction: “people aren't convinced yet? Well what if I also told them that ….....”

So here's my challenge to you: the next time you want to test one of your pages (which you should be doing right now), don't change elements and don't add new ones. Pick 3 of the less-important “things” on the page and delete them. That's your test.

Here's how I suggest you do it:

  1. Look at your page, think about which 3 things are most important. Usually that's the 3 W's: what is this page about, why should a user care, and where are they supposed to go next? If you can't narrow this down to 3, you can cheat and pick 5 things. Not more.

  2. Go to http://www.eyequant.com and analyze the page. If you haven't used EyeQuant before, your first test is free and your results will be ready in seconds.

  3. Look at the “Attention Map” - that's the one that looks like a “heat” map. Excluding your 3-5 key content pieces, anything that has some red/orange/yellow overlay on it is a great candidate for deletion. Remember, you have to delete 3 things!

  4. If your page now looks ridiculous, make any small layout tweaks that might be necessary. Try not to do anything radical though.

  1. Run A/B test.

Did it work? If you try this, I'd love to hear from you. In fact, if someone has interesting results that we can share on the blog, I'll hook you up with another 10 EyeQuant analyses! Just shoot me an email at kurtis@eyequant.com or tweet me at @kurtiswmorrison.

Designers & Growth Hackers: Show Your Process

Show me, don’t tell me: visually communicating your process eases tensions across departments, and will make your design meeting a hell of a lot easier.

At Media Evolution’s The Conference last week in Malmo, EyeQuant’s co-founder Fabian Stelzer spoke about the war between data and creativity that so many of us feel in the marketing world, and offered a few concrete tools to make peace between these two perennially opposing sides.

Most of the way through the talk, he made mention of a simple yet universal way of bridging the conversion optimization language gap:

Show your process.

As opposed to an analytics expert slamming a 42 page market report on the conference table, or as opposed to a designer shrugging their shoulders and saying, “just trust me, I’m a creative”, showing how you’ve worked through a problem is crucial to finding understanding between teams. In this way, a universal, visual language is formed.

In this post, we’ll take a look at 5 key ways to visually explain your process, find common ground between different departments, and work through your web optimization process more efficiently.

Here we go...

A macroscope for Big Data

Back in 1979, Joël de Rosnay began speaking about a crazy idea called the “macroscope”. If microscopes are for observing the infinitely small and telescopes are for observing the infinitely great, thought de Rosnay, then what kind of instrument could we use to observe the infinitely complex? For de Rosnay, this was the macroscope: a global, holistic view of the world around us - one that finds connections through infinite detail.

MACROSCFIG1-438x235 (1)

With a global view of a problem (like a bird’s eye view from space), wouldn’t we then be able to see the best paths to take to solve our problems?

This, of course, is the rhetoric that has played through the minds of everyone who’s ever uttered the words, “big data”: in a complex world, we need tools to help us cut through the brush and see the light at the end of the problem: not just numbers, but connections.

We often forget to take into account a macroscopic view of the various problems that arise when it comes to web optimization. As a result, we get caught up in details, start fidgeting with pixels, and forgetting that, by zooming out of the minutiae, there’s always a big picture.

Create a Mental Image of Your Strategy

Chris Spooner made this great list for Line25 about design agency websites that do a good job of visually describing their research and design process. His examples vary from venn diagrams to squiggles, but the end result is clear: visual descriptions of complex strategy provide a mental image of a problem and its solution, putting everyone at ease by graphically describing what a process entails, why a certain part of the process is important, and most importantly, and by giving a macroscopic view of what the designer will do to solve their client’s problems.

Bonus: a global mental picture of your research and design process can help to justify aspects of your budget that might otherwise get cut out.

Screen Shot 2013-08-30 at 2.46.05 PM

Visualize your data

When we first set out to create EyeQuant in 2009, we had a hunch that our potential customers - e-commerce companies and digital agencies - were feeling the tension between data-driven and creative design.

Designers already had the experience to know what might work, but increasingly we found that they would need to back up their findings with data - in a way that everyone (including themselves) would understand. After all, marketing and advertising play in a far bigger playground these days, and intuition alone doesn’t cut it.

Crucial to ending the war between data and creativity is learning how to blend the two together. What we needed are tools and strategies to visualize data, and vice versa: accurate, intuitive, yet complex enough tools to grip the big picture - and all the little ones that surround it.

In the world of data visualization, Moritz Stefaner, who works with everyone from FIFA to Skype, is one of the great minds. In this lecture, Moritz makes an astounding point about marrying information with design - and leaves clues to everyone from growth hackers to designers on how to work with and show data and process:

“good visualizations show you the data, great visualizations show you the patterns of the data . . . good visualizations answer questions, but great visualization generate new questions . . . good visualizations tell a story, but great visualizations tell a thousand stories”.

Share your prototypes

There is a persistent misconception outside the design world that creatives tend to pull ideas out of thin air a couple minutes before a presentation. The reality couldn’t be further from the truth, but if this isn’t shown, then nobody will be the wiser. From an initial sketch on a napkin to a full deconstruction and reconstruction of a competitor’s web page, showing visual examples of each iteration of your design process provides transparency, and proves that you’re thoroughly working through problems.

A fantastic resource for keeping track and sharing each step of your design is LayerVault. LayerVault is a beautifully designed software that saves and catalogues every single revision you make, and let’s you decide which iterations to show to colleagues. What makes this app really special is the ability to return to a step you’d written off a day ago that, the next day, turns out to be the solution (without pressing command_z a thousand times).

Show your mistakes (and the tools you used to solve them)

We know, we know, there is a certain level of ego that needs to be upheld during design meetings in order to maintain a modicum of authority. As the old saying goes, though, “the truest characters of ignorance are vanity and pride and arrogance.” Speaking about your mistakes (and better yet, showing them) promotes an environment of empathy, of teamwork, and of humility. We’re all human, we all make mistakes, so let’s ‘fess up to them and learn from them.

Of course, making informed iterations during the design process can be difficult; testing a re-design or a new web page pre-launch is next to impossible, and going on market or design research alone leads to many unknowns.

  • Usability Hub is a crowd-sourced app that allows you to test several aspects of your design - first impressions, click engagement, and navigability - through quick user surveys. Usability Hub provides easy-to-read reports, word clouds, and test answers that give invaluable pre-launch insight into what you’re doing, and why.
  • Then there’s us, EyeQuant (we know, we know, self promotion...but indulge our vanity for a few seconds, we’re just really excited about what we do). Our idea is simple: At any stage of the design process, a screenshot can be fed into EyeQuant’s online app, and within seconds three visualizations of user attention will appear. We’re the only predictive eye-tracking software on the market with a strong neuroscience background, which is why we can safely say that our technology is over 90% accurate in comparison to a traditional eye-tracking study.  Oh yeah, we have a patent too. Here’s how it what it looks like when we did an analysis of ING’s website.

Come together, right now

Showing your iterations, showing how you tested them, and then showing what went right and what went wrong is a sure-fire way to impress upon your boss, your team, and your clients that you know what you’re doing. Laying out a clear roadmap not only instills trust in the others at the meeting, it also lets designers work with far more daring ideas. Just look at Google.

Better yet, it works both ways:

  • As a designer, showing your process proves to clients and colleagues that you’re not afraid to test your intuition and to work with tools to make this happen.
  • As an analytics expert, learning to work with information visually (and creatively) shows everyone else that data is not a cold clump of steely percentages.

During his talk at The Conference, Fabian suggested that we need tools that provide “creative data” - tools that are fast, communicative, objective, and not all too final. This last point is crucial: both creative and growth hackers need to come to terms with the limits of their own domain, and this is where they can come together.


Ending the War Between Data and Creativity: Big Ideas from Media Evolution's The Conference (Video)

Last week in Malmö, Sweden, innovators like Reddit co-founder Alex Ohanian, branding expert Cindy Gallop, and EyeQuant's very own Fabian Stelzer convened to speak about the future of design, technology, and communications at Media Evolution's The Conference. During his presentation, Fabian walks us through a defining question for many at The Conference - an issue which has integral to any discussion about branding, advertising, and marketing. That is, how do we reconcile data with creativity? 

Reaching into philosophy, into art history, and into science, Fabian fleshes out a new way for web designers and online marketers to think about testing, analytics, and making decisions by using data creatively.

Clocking in at about 15 minutes, you can check out what Fabian has to say during your coffee break:

Meanwhile, if you've got some more procrastination time to spare today, we highly, highly recommend checking out our favourites from The Conference:

Cindy Gallop on redesigning business, sex, and opportunity:

Matthew Berman, on distribution hacking:

...and for something more unexpected, here's James Bridle's talk on mental and conversational models of new technologies:

Lame Optimization Ideas Hold Up Design Meetings: Here's How to Stop Them

There’s an age-old saying that goes, “The only thing that stands in the way of a business making more money is a design committee meeting.”

Too often, a vague sense of compromise seems to be the only thing that is accomplished during a website optimization or re-design meeting. In the end, everyone is left with the feeling that a melon-baller has been taken to their heads, drained by the knowledge that that nobody really knows what they’re talking about, nor precisely how to fix what needs to be fixed.

So, how can you turn a design meeting into a painless and (most importantly) fruitful experience?

In this article, we pick apart the 3 lame optimization ideas that tend to hold up design meetings, and provide strategies that will put everyone at ease.

Lame Idea #1). “We should make the call to action [insert trending color] because I read a blogpost that says that it will guarantee great conversions”

Underlying reason for lame idea: Insecurity that leads to blindly following the pack; esoteric conversion thinking cajoled by addiction to content marketing blogs.

It’s unsurprising that someone is using their knowledge of the online marketing blogosphere to corroborate their opinion. Following the advice of experts and the gurus is common practice (and can provide actionable ideas), but it’s essential to be aware of contextual as opposed to catch-all ideas:

Screen Shot 2013-08-15 at 11.01.08 AM

Catch-All Idea: There is little measurable data out there to suggest that any single color converts better than another. Suggesting that a call-to-action button should be changed to orange, or green, or purple just because a blog says so falls into the catch-all idea category, where someone is looking for a quick win over an educated decision.

Screen Shot 2013-08-15 at 11.00.04 AM

Contextual Idea: Suggesting that the call-to-action button should be changed to orange because it will contrast well with a pale blue or green background qualifies as contextual idea. Extensive research, including this study from the California Institute of Technology (Caltech), show that user attention is initially attracted to the most salient areas (or areas that “visually pop out”) on a page, especially under time constraints or while attempting to multi-task. Saliency doesn’t necessarily occur from one, brightly colored element, but from the contrast and spacing that surrounds it. Place an orange CTA in front of a yellow background, and it will get lost. For that matter, not even a flashing yellow “BUY NOW” button will catch user attention when it’s set against a cluttered page.

There’s only one way to verify whether an idea garnered from a blog will work: test it.

  • Once you’ve implemented your changes, run an A/B test to see how well your ideas stand up to alternatives.

Lame Idea #2). “Look, we’ve spent a lot of money on advertising for this product, so the campaign video/branding need to be front and centre on the landing page”

Underlying reason for lame idea: Branding team worried about the hard work they’ve already put in to the project; lack of understanding as to what a landing page is actually supposed to do.

E-commerce companies in particular tend to place an inordinate amount of importance on branding and advertising on landing pages as opposed to showcasing the core benefits of the product on a landing page. The rationale behind this tends to be that an emphasis on branding will link the product’s online presence with its offline presence.

Unfortunately, landing pages just don’t work like this.

A landing page should clearly communicate what the product is, why it’s useful and innovative to a potential customer, and where the customer can go to learn more - all within a few seconds. Cluttering a page with expensive ad campaigns merely serves to distract from the core value of the product in question. Confuse your user, and they’re far more likely to bounce before they look for a button to click.

Visual examples of user behaviour often succeed in ending branding discussions when rhetoric and statistics fail. Predictive Heatmaps like EyeQuant's show where a user will look when they first arrive, objectively (and accurately) illustrating why overbearing branding confuses users and drives them away from completing a task, while also providing great insight into what could be changed.

Bonus: They can be generated in a few seconds.

Here's an example: The landing page for the 2013 Volkswagen Beetle tells us an awful lot about how the car is being marketed, but when a user first arrives at the page, they miss its core benefits, pricing, and where a prospective customer can go to learn more:

As a comparison, the landing page for the 2013 Ford Focus still showcases a slick photo shoot, but from the heatmap below, it's easy to see that the product's core value, it's price and fuel efficiency, are immediately visible to users:

Lame Idea #3). “We can’t seem to decide what the most important product/value on offer is. Oh, I know, let’s use a scrolling banner/carousel, then everyone wins.”

Underlying reason for lame idea: Meek attempt to please everyone in the room via indecision; futile addiction to web design gimmicks.

Here’s the short response:

Nobody wins with scrolling banners/carousels.

Jared Smith sums up the reasons why you should never, ever resort to carousels or scrolling banners on your landing page incredibly well with his Should I use a carousel?.

Like this:

Screen Shot 2013-08-14 at 2.39.53 PM

Or like this:

Screen Shot 2013-08-14 at 2.39.53 PM

 Choosing a carousel is approximately tantamount to telling your customers, ”we can’t make up our minds about what we’re selling, so can you just do it for us?”. Carousels are confusing, they crash frequently, and in an attention-starved online world, they have the uncanny ability to make people back away and never come back.

Just ask Adam Fellowes, head of UX at Digirati:

Screen Shot 2013-08-14 at 3.05.37 PM

Fighting the good fight

Design meetings get out of hand when those present believe that their ideas are more important than anyone else’s. The result of this is narrow-thinking and an overarching fear of listening to what others have to say. When this is the case, adding a data-driven yet visually intuitive example tends to work very well to dispel tension.

After all, at the end of the day, success feels way, way better than compromise.