Seonglae ChoNormalized Discounted Cumulative Gain
A normalized metric that considers both ranking and scores of correct answers
Both binary and numeral scores, Normalized metric, Hard to explain
Ideal DCG
best possible DCG result (the perfect ranking) for the query. Perfect ranking is the one sorted by the relevance score. (Not only support binary relevance)
NDCG@K
K means the top K ranked item of the list, and only top K relevance contributes to the final calculation for computational efficiency
LambdaRank loss
Multiply actual gradients with the change in NDCG by swapping the rank positions of the two documents for Listwise objectives
Cumulated gain-based evaluation of IR techniques | ACM Transactions on Information Systems
Modern large retrieval environments tend to overwhelm their users by their large output. Since all documents are not of equal relevance to their users, highly relevant documents should be identified and ranked first for presentation. In order to develop ...
https://dl.acm.org/doi/10.1145/582415.582418
DCG
Discounted Cumulative Gain
Discounted cumulative gain
Discounted cumulative gain (DCG) is a measure of ranking quality in information retrieval. It is often normalized so that it is comparable across queries, giving Normalized DCG (nDCG or NDCG). NDCG is often used to measure effectiveness of search engine algorithms and related applications. Using a graded relevance scale of documents in a search-engine result set, DCG sums the usefulness, or gain, of the results discounted by their position in the result list.[1] NDCG is DCG normalized by the maximum possible DCG of the result set when ranked from highest to lowest gain, thus adjusting for the different numbers of relevant results for different queries.
https://en.wikipedia.org/wiki/Discounted_cumulative_gain