Comparing complex multi-variable systems, especially for understanding market perceptions
A process for comparing the effects of multiple components of a website in a live environment. This is essentially many comparison tests performed on one page at the same time.
A dynamic website can be used to serve up different versions of a page in equal proportions to visitors. The key is then to gather the statistics on how all the different elements performed and what each visitor went on to do after seeing the elements under test. With many variants this can be a complex statistical challenge.
Numerous factors impact customer behavior; this process can be used to test alternative text, form layouts, images, background colors, and so on. Not all elements have the same impact, so by comparing results from different tests, it is possible to identify the combination of elements that consistently produces the most desirable effect.
Carrying out multivariate testing on sites with low traffic can be time consuming as it then takes a considerable time to expose a significant number of people to each variation.