Relevance is Irrelevant

Without fail, a company will employ a recommendation engine for a purpose (nobody does this for fun, really). Often, that purpose is profit (or something along those line). For most companies, ‘relevance’ is irrelevant (no pun intended).

The success of any recommendation engine should (in my opinion) be measured by its ability to meet the objectives it was intended to achieve. As said, in most cases, this will be tied to sales or profit.

A/B test your system against a control (often random, might be rules). If your recommendations increase sales (or decrease costs, or decrease call handeling time, or increase revenue, or increase customer satisfaction, etc) compared to the alternative you’re doing pretty good. You can forget about the rest.

Who cares about relevance if you can measure business value?

[ Posted on Quora as answer to What is the best way to test the relevance of a recommendation engine? ]

Art-Rock Allegory

Contrary to what it may seem like, I have been busy writing; just not for this particular blog.

[ Crossposting from Quora where I tried to answer the question “Why hasn’t anyone ever seen an an animal give birth to an animal of a different species?” ]

An answer in an art-rock allegory.

Imagine you are carving a statue out of a huge chunk of solid rock. Before you begin, the rock is just a rock. After you are done, the rock is gone and replaced by a marvelous (one would hope) piece of art.

As you are chiseling away you start to wonder: with which exact blow of the hammer does the rock suddenly transform from stone into sculpture? Why do we not suddenly see art appear where there was only a boulder before?

Individual animals are like chips falling to the ground as mother nature patiently carves up new marvels. Species are imaginary constructs we use to group individual pieces of the puzzle, but they can only hint at the true nature of the masterpiece they were hewn from.

When considered up close, each animal is neither rock nor art.

Art rock.

The Middle Way

James Taylor is spot-on.

Too many analytic professionals think that only the data speaks and that business rules are, as someone once said to me, “for people too stupid to analyze their data”. Similarly too many IT professionals think that everything can be reduced to business rules or to code using explicit analysis. The reality for most decisions is somewhere in between.

In order to truly achieve business transcendence one must follow the Middle Way.