Recommender Satisfaction

Model evaluation

• IR System Evaluation

• Confusion Matrix

Behavioral metrics:

• Clicks, Scroll, Zoom, Swipe, Likes, Shares

• Dwell time, session length, Inactivity, Transitions

• Gestures, eye tracking

Feedback

• Item Level Feedback

• Page Level Feedback

• Session Level Feedback

• Intra-Session Feedback (loyalty /
  Cohort Retention )

Feedback Types

• Explicit Feedback - annotator & survey

• Implicit Feedback - understand user engagement from behavioral metrics

Final: Satisfaction Prediction (SAT Prediction)

NeurIPS 2020 : (Track2) Beyond Accuracy: Grounding Evaluation Metrics for Human-Machine Learning Systems

Please be aware that videos might show the previous tutorial in the same track.

https://neurips.cc/virtual/2020/protected/tutorial_a9588aa82388c0579d8f74b4d02b895f.html