Tag: ab testing

Using A/B testing to prioritise performance optimisations

Back in December 2015 I spoke at LDNWebPerf alongside Peter Chamberlin about how web performance is not a technical problem (slides with speaker notes). One of the things I talked about was how we used multivariate testing (MVT) at BBC News to prioritise performance optimisations. The gist of it was that our stakeholders had already bought into the idea that performance has a strong correlation to business metrics, and they wanted to dedicate some development time to improving performance. The catch was that they didn’t want to spent too much time on it.

Our predicament, then, was that we needed to know which optimisations had the biggest impact on performance without actually spending the time to make the optimisations. For example, we had a hunch that inlining the critical rendering path CSS would improve our start render time, but with over 1MB of CSS and a complicated application architecture, implementing this was much easier said than done.

This is where the idea to A/B test performance came from: we could easily make the performance optimisations by hand on a single page, and then benchmark each of the optimisations to find out which had the biggest impact.