Seonglae ChoHuman-designed knowledge, heuristics, and structured algorithms may look good in the short term, but eventually their progress stalls. Simple general learning methods that can scale with computational resources ultimately outperform all human knowledge-based approaches in the long run. This is why it's called the "Bitter Lesson" = human knowledge and design interventions are inherently destined to lose.
Don't rely on human knowledge or design; general algorithms that learn on their own using computational resources ultimately win. Data and computing always wins heuristic algorithms.
- Computing power matters
- The great power of general purpose methods, of methods that continue to scale with increased computation
