Seonglae ChoSelf-supervised learning (self-directed learning) is considered the efficient method for generating intelligence. Supervised learning have low efficiency in forming intellectual information through trials. Children also build their world model through observation rather than supervised learning.
Self-supervised learning defines a pretext task based on unlabeled inputs to produce descriptive and intelligible representations (Balestriero et al., 2023). This approach enables AI systems to learn from limited data by systematically generating and learning from self-created training examples.
Key characteristics:
- Used in Bidirectional LM for pre-training Masked Language Models
- Allows AI to learn efficiently from small datasets through self-generated systematic data
- Example: From a single dog image, AI can:
- Generate multiple rotated versions
- Automatically extract and learn from distinctive features