* Denotes required field


Tuesday, 20 March - 09:04 (Category: e-marketing, Formulae for Profits )

A/B Testing is Dead! Long Live AB/C

SYNOPSIS: A/B testing can be augmented by introducing a third component. This third, completely different, test provides a normalising function proving whether the initial A/B idea was any good in the first place.

Last night my friend Chris and I were having a drink together. He works for a major retail business which is just about to launch its online shopping experience. Chris and I were chatting about this and he casually mentioned, "We've got some redhot consultants and they've come up with some cracking ideas. One of them was something called A/B testing where they test two variations of the webpage to see which generates the most profit."

In a fit of defiance and not to be outdone by "redhot consultants" I said, "Ha! Old hat. Marketeers have been using that technique for over 100 years now. There's nothing special about split testing. But I bet they've not proposed AB/C testing?"

(At this point even I didn't know what AB/C testing was but having successfully painted myself into a corner there was only one way out -- a spot of quick creative thinking!)

So what is AB/C testing?

Let’™s start with A/B testing which, in marketing terms, is really very simple. Let's say you're running an advertising campaign and the headline is...

Ladies Elegant Dresswear Sale

You won't know whether this is the most effective headline unless you run a slightly different one and measure the comparative results. The important thing about A/B testing is you only change ONE thing at a time. So your second headline might be...

Sale of Ladies Elegant Dresswear

If you change more than one thing at a time you won't be able to identify which of the individual things you changed made the difference. The problem with this technique is threefold:

  1. It assumes your initial idea was any good in the first place;
  2. you can only make improvements by small degrees and
  3. there will always be a 'ceiling' of success which you cannot improve upon.

This brings us onto AB/C testing. A good deal of marketing is creativity so it is essential that you test TWO creative ideas rather than two variations of ONE idea. However, A/B testing only allows for the latter, therefore, you need to do is add a second creative idea (or C) in to the testing mix. However, it is a completely fresh and different idea. For example...

10% off Ladies Dresswear

By introducing a completely new idea (C) you can test whether it is a fundamental improvement on either A or B. If our C test is more successful then you split test it (i.e. it becomes the new A/B). That would then leave us with the option of creating a new C test and maintaining two different ideas at the same time.

By adding C into your split test system you give yourself the ability to improve your marketing by orders of magnitude rather than increments.

(A big thank you to Chris for making me think on my feet!)

Posted By:Jed Wylie at 09:04, 20-03-07