Far from killing creativity, A/B testing can unleash it, argues Phil Williams, UX designer at PRWD.
Phil Williams
It's methodical, calculated and many believe, unimaginative. A/B testing is a scientific process which draws insights from vast amounts of quantitative and qualitative data. We draw hypotheses from this data, make improvements and set it against the control as a test. But to what extent does testing kill creativity in our industry?
The truth, I believe, is that A/B testing your site actually encourages a far greater level of creativity, since it makes us ask that all important question: 'What if..?' What if we removed that image, flipped the layout, or changed the process completely?
These are fundamental questions often only asked after a session of rigorous user testing. After we've asked the all-important 'if' question, we come to the 'why' and the 'how'. For example, 'why should we alter the site map, and how should we do it?'
Asking these questions enables creativity to enter into every stage of the optimisation process, as creative teams are less afraid to voice any questions or ideas over fear of upsetting the status quo.
Think back to childhood. You could ask those 'why', 'if' and 'how' questions on a regular basis without fear of judgement. In the working world as an adult, to ask a question like this might seem far too daunting or risky.
This is the reason why questioning your website or challenging a particular process can lead to whole new world of discovery, change – and creativity.
Avoiding those pink elephant ideas
The creative process on a basic level is lateral thinking in its simplest form. We are presented with a problem and we wrack our brains to try and think of something else. Let me demonstrate:
Please don't think about a pink elephant… What are you thinking about now? Is it is a pink elephant?
Although I asked you not to, I'm betting that right after I simply mentioned 'pink elephant' that's all you could think about. When we are presented with an obstacle, our brains try to find a way around it which often sends us straight back to square one.
Illustration by Mike Chipperfield of Magictorch: www.magictorch.com
With A/B testing, our control works like this pink elephant. As a designer, my job is to think laterally rather than just changing the colour – it can be completely the opposite, like removing it completely.
Great creativity isn't about making the flashiest website with the most aesthetically pleasing graphics. My ethos is against designing for design's sake. I look at a problem, and figure out a way to deliver maximum results with making as few changes as possible – much more creative if you ask me.
Proof that it works
An A/B testing exercise was undertaken for EA Game's Sim City 5. The objective was to increase the number of pre-orders for the game. The control (on the site already) offered the reader $20 off their next purchase via an attention-grabbing banner.
The original creative process is clear to see here – draw attention to a great offer and more people will click through to buy the game. It seems obvious, but the results tell a different story.
Surprisingly, the simpler version without the promotional banner actually drove 43.4 per cent more purchases. The banner was having the opposite effect than intended.
Results
What the data drawn from this A/B testing tells us is that users just wanted a simple way to buy the game, and instead of enticing people to buy, the banner was actually acting as a barrier to purchase. It might be argued that the web page without the banner is less creative, but in this case, the decision was taken to remove it completely.
After pinpointing where the problem areas on your site are, it is creating new hypotheses to test that can truly allow creative thought to flourish. Crucially, it's about keeping your mind open to new possibilities for how things could be. A/B testing simply provides an audience to validate one creative choice over another. A/B testing doesn't kill your creativity, it adds fuel to it.
Words: Phil Williams
Phil William is a UX designer at conversion optimisation agency PRWD.
