Orca - Microsoft Research
Redefining small LMs performance In this project, we develop technologies for creating, improving, and specializing small LMs (~10B parameters or less). Our research involves self-improvement strategies, feedback-driven teaching methods between large and small models and utilizing domain specific data to specialize LMs. We focus on using richer training signals for teaching small LMs to do […]
https://www.microsoft.com/en-us/research/project/orca/
