
There is a dirty little secret in the testing world called sample size pollution. Pollution of your sample audience can unknowingly cause tests to be doomed before they even start. Sample size pollution can be split into 4 common types: timing, device, browser, and cookie. Your experimentation strategy and power of your software will make the difference in how well you minimize sample size pollution. [OPTIONAL] To make sure you have a representative sample, Convert suggests running a test for at least one business cycle. For a valid test, make sure it reaches your calculated sample size for your desired MDE (Minimum Detectable Effect). Sample size pollution is a negative by-product that’s experienced when you run A/B tests, but using the right tool and the right strategies will help you mitigate the risks. Here are our tips or strategies to help you reduce sample size pollution: