A sampling method based on forecasting and combinatorial optimization for high performance A/B testing

摘要

1 Introduction A/B testing[1]is a popular statistical method for many applications,especially,widely used by Internet companies for user experience research.The sampling algorithm is an important part of A/B testing and has a critical impact on the performance of A/B testing.There are two challenges to achieving high performance in A/B testing.(1)The features of individuals being sampled change over time and the sampling results used for evaluation are valid only for a period of time.(2)The sampling methods in A/B testing require determining whether the two groups of samples have an identical distribution while Simple Random Sampling(SRS)has a different purpose,which only concerns whether the sample distribution is identical to the population.