System comprising: one or more computers and one or more storage devices having stored thereon instructions that are adapted to, when executed by the one or more computers, cause the one or more computers to perform operations comprising: Providing value-based training data, the training data comprising multiple instances of (i) user characteristics, (ii) characteristics of corresponding content elements that are presented to users, and (iii) respective values of a value variable that are determined for each of the content elements for the users wherein the respective values of user behavior with respect to content items presented to the users are determined by a value model based on an anchor variable in the user behavior; Training a scoring model on the training data to generate score-based scores from user features and content item features; Rating a plurality of candidate content items selected by a rating engine for a first user, the rating engine receiving respective value-based scores generated by the trained rating model for the candidate content items and the first user; and Providing two or more of the content elements in question for presentation to the first user in an order determined by the classification.
展开▼
