Optimizing the Mapping of Qualitative Labels to Scores for Calculating Gain in Search Results

摘要

In one embodiment, a method includes receiving, from a client system, qualitatively-labeled search results, determining, based on an initial mapping scheme that maps the qualitative labels to an initial set of scores, an initial score for each of the search results, calculating an initial normalized discounted cumulative gain (nDCG) for the search results, generating a new mapping scheme that maps the qualitative labels to a new set of scores, and includes one or more pairs of non-consecutive scores, where a new nDCG calculated for the new search results is greater than the initial nDCG, determining, based on the new mapping scheme, a new score for each of the search results, and generating a new ranking algorithm based on the new set of scores, wherein the new ranking algorithm ranks the search results to improve the nDCG for each set of qualitatively-labeled search results.