Seonglae Cho
Need good offline and check online correlation
Consideration
- Production recommender and test recommender’s logged feedback might different
- Special logged data need to be collected through randomized data or log propensity scores
- Counterfactual evaluation → off-policy evaluation
Offline Evaluation to Make Decisions About PlaylistRecommendation Algorithms - Spotify Research
Spotify’s official research blog
https://research.atspotify.com/publications/offline-evaluation-to-make-decisions-about-playlistrecommendation-algorithms/
Offline comparative evaluation with incremental, minimally-invasive online feedback - Spotify Research
Spotify’s official research blog
https://research.atspotify.com/publications/offline-comparative-evaluation-with-incremental-minimally-invasive-online-feedback/
Estimating clickthrough bias in the cascade model - Spotify Research
Spotify’s official research blog
https://research.atspotify.com/publications/estimating-clickthrough-bias-in-the-cascade-model/