Forever Learning

Forever learning and helping machines do the same.

A/B Testing XXL

with 4 comments

[I’ve tweeted about this before.]

If fashion stores believed in A/B testing, they would probably only sell white XXL shirts. Most customers would fit tent-sized garments; most colours go well with white. Giant colourless shirts would presumably have the better sales conversion rate by far.

But of course this would be far from optimal.

Customers come in different shapes and sizes. If you really want to maximise conversion, you will have to tailor to their specific needs and personal preferences. A/B testing might be the latest fashion, but the truth is that some customers will have a taste for B even though the majority might fancy A. This is why these 20 lines of code will beat A/B testing every time.

The trick is not to figure out whether A is better than B, but when A is better than B; and for whom.

Marketing should not be one-size-fits-all.

Written by Lukas Vermeer

November 30, 2012 at 18:05

4 Responses

Subscribe to comments with RSS.

  1. Definitely a good point!

    One of the things I appreciate the most about Facebook is their ability to come up with a different site depending on who is using it:
    if you have few friends, they will strengthen the features that help you discover more,
    if you do not check it often, they will rely on email notifications,
    if you stop using it for a while, they will try to bring you back.

    I was amazed when I realized that Facebook looks quite different depending on whether it’s me or my 14 years old cousin using it.

    Alessandro Bahgat

    December 16, 2012 at 20:47

    • Interesting! I did not know that. Everyone knows they filter wall posts based on what you’ve shown interest in before, but I was not aware that they actually change other aspects of the experience. Thanks for sharing.

      Personalisation cleverness to drive engagement aside, I still think Facebook is not in a good position to monetize all those eyeballs. Attention is virtually free to give, but a company cannot survive on attention alone. More here:

      Lukas Vermeer

      December 17, 2012 at 10:39

      • Agree on that one: while I see some value in their product, the current approach they have to advertising does not work for me.
        I believe they could have some good possibilities in brand reputation, but only if they manage to get rid of the hunt for likes that characterizes many Facebook brand campaigns.

        About post filtering, ever heard of the Filter Bubble? Just like overfitting, but with consequences on society.

        Alessandro Bahgat

        December 17, 2012 at 11:51

        • There’s a TED talk about the filter bubble I watched a while ago. I must say I understand the potential problem, but still prefer computer curated content over editorial dictatorship. Previously, news was selected for me by an army of editors based on what they though was important for me. Now, computers are starting to take the editorial bias out of the equation.

          You can never read everything. You will always need a filter. There will always be a bubble.

          Yes, there are risks; but overall I think this is a good thing. The filter bubble is not new, it’s just no longer controlled only by people. I believe the use of algorithms provides unique opportunities improve the way in which we filter our information diet.

          Lukas Vermeer

          December 18, 2012 at 10:38

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: