METHODS AND SYSTEMS FOR GENERATING VALUES OF AN OMNIBUS EVALUATION CRITERION

摘要

A method and server for generating values of an overall evaluation criterion (OEC) to be used during a control/treatment testing is provided. The method comprises, for each of the control and treatment versions: receiving datasets indicative of user interactions; determining first sets of values of a first per-user metric; and determining second sets of values of a second per-user metric. The method comprises selecting a linearization value. The method comprises, for each one of the control and treatment versions, determining linearized sets of values where values in a given linearized set are differences between (i) values of the first set from (ii) values of the second set that are weighted by the linearization value; and generating OEC values based on the linearized sets. Difference between OEC values is indicative of a magnitude and direction of the change in interactions between the versions.