Mutlivariate Testing

A/B tests are usually performed to determine the better of two content variations, multivariate testing can theoretically test the effectiveness of many many combinations instead.

Multivariate Testing could mean that we not only have two variations but we can test simultaneous many features of a page or process at the same time.

Multivariate Testing Vs A/B Testing

Now, you can test only one change at a time by a single A/B test or test many changes simultaneously by a MVT!! The upside for A/B is that it isn't complicated to build and run A/B tests; it is really good for redesigns, say a new home page or checkout process; and you can get results fairly quickly if you have enough site traffic. The downsides are that everyone has variations in traffic. The time of day, day of week, seasonality, different promotions, TV adverts, search traffic are all factors in your traffic. Running A/B tests at two different times may get completely different results. On the other hand, MVT allows you to try lots of potential replacements for these elements. You can test large amounts of variables at a time; theoretically millions of them, practically all tools have a limit to 16 variations!!You can also design tests across several pages which lets you obtain serious improvements to signups, sales, checkouts!!

Which one is good for me?

If you're new to testing start to crawl before you walk or run. Google optimiser is a good start and if you need something more advanced then you can start think of paid solutions!!

An A/B test can be good for basic comparative testing. While multivariate testing is good for larger scale optimisation. Depending on your needs there are free software like google optimiser or paid solutions like test and target from omniture or content MVT from Maxymiser!!

In any case, testing is highly recommended as will increase you conversion rates and boost your bottom line!!

I reckon start testing immediately!! The clock is ticking, tik tak, tik tak, tik tak!!

Related Videos: Avinash

Related Posts: WedAnalyticsDemystified and Avinash’s post