YSU NLP Midterm

1시간 주관식

• Loss Function*

• One Hot encoding*
• Orthogonality 로 내적하면 0
• Curse of dimensionality

• Dimension Reduction

• AutoEncoder encoder block이 중요

• Distributional semantics*

• Word2Vec*
• CBOW*
• Skip-gram

• Autoencoder와 CBOW의 차이점*
• just lookup CBOW는

• Sequential Data

• RNN Process Sequences

• Recurrence formula makes RNN be aware of temporal dependency while the Feed-forward NN has none for new state from old state*

• Text Classification
• Sentiment Analysis
• Lexicon-based Approach
• Machine Learning Approach

• GLUE, SuperGLUE, KLUE

• Transfer Learning
• Cross-lingual Learning
• Multi-task Learning
• Negative Transfer

• Confusion Matrix,
  Accuracy,
  Recall,
  Precision

• Precision-Recall tradeoff
• F1-score

• Attention Mechanism
  Cross-Attention
  Encoder-Decoder Model*

• Reading Comprehension
• attentive reader bidirectional LSTM, ODQA

• Transformer Model 3 identity**
• self
• parallelism
• multi head

• Positional Embedding*
• 두 단어 사이의 거리가 멀수록 해당 단어들의 포지셔날 인코딩 간 거리는 멀어야됨
• 모든 위치에서 인코딩 벡터의 크기는 동일해야

• Evaluation

• GPT 2

• GPT 1