A/B Split test—hey, no fair!

AB Split test

I’m a big fan of Which Test Won and if you have a website and haven’t signed up yet, I would encourage you to do so.

And now for the cautionary tale…

In a recent test they showed and A/B test that was designed to see if humor was more successful than just the facts. Awesome test right? The editors pointed out that humor was always tricky, which is true, but they didn’t point out that the primary drivers were different for each test. I’m not sure humor had anything to do with which test won. Below is the headline and subhead copy for each test. See if you can spot the crazy big motivators that should have been included in both tests to make them a valid A/B split test on humor.

Here is test A:
Did your iPhone Screen Have A Bad Night Out?
We can’t cure a hangover, but we can make your iPhone look brand new in less than an hour for half of what the other guys charge.

Here is Test B:
iPhone 4 or 4S Screen Repair
Our Promise: Fixed Fast. Fixed Right.

Sorry kids, but ain’t no way that “Fixed Fast. Fixed Right” is going to trump “Fixed in one hour for half of what other guys charge”. You can say it any way you want to but those are two insanely huge benefits up against a puny and somewhat vague guarantee. It was never about humor and it was never a fair test.

Did you enjoy this post? Sign up for more.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

The Catalog Blog covers opinions and information on all things catalog. Have something to add? Leave a comment below. Catalog University is devoted to helping you get ahead in the fun and fascinating world of catalogs. If you want even more information about cataloging, including FREE Pub Talks, be sure to sign up for the Cat-U mailing list. We will never share your name with 3rd parties.

Sarah Fletcher
President/Creative Director Catalog Design Studios Inc.
Co-Founder Catalog University