Recommender System Deployment

Consideration

• Production recommender and test recommender’s logged feedback might different
• Special logged data need to be collected through randomized data or log propensity scores
• Counterfactual evaluation → off-policy evaluation

Offline Evaluation to Make Decisions About PlaylistRecommendation Algorithms - Spotify Research

Spotify’s official research blog

https://research.atspotify.com/publications/offline-evaluation-to-make-decisions-about-playlistrecommendation-algorithms/

Offline comparative evaluation with incremental, minimally-invasive online feedback - Spotify Research

Spotify’s official research blog

https://research.atspotify.com/publications/offline-comparative-evaluation-with-incremental-minimally-invasive-online-feedback/

Estimating clickthrough bias in the cascade model - Spotify Research

Spotify’s official research blog

https://research.atspotify.com/publications/estimating-clickthrough-bias-in-the-cascade-model/