AI Alignment

Creator

Creator

Seonglae Cho

Created

Created

2020 Aug 23 9:36

Editor

Editor

Seonglae Cho

Edited

Edited

2024 Nov 21 20:59

Refs

Refs

Alignment Problem, AI Control

A Maximally Curious AI Would Not Be Safe For Humanity

AI is aligned with an operator - AI is trying to do what operator wants to do

Aligned doesn’t mean perfect (Controllability, reliability)

가르친 행동과 다른 행동의 불일치 정렬

모델의 능력보다 정렬이 더 빠르게 발생해야 한다

신경망의 내부를 보고 해석하는 다른 신경망이 필요할 것

AI Alignment Notion

Natural Abstraction Hypothesis

Outer Alignment

Inner Alignment

Mesa Optimization

Stop button problem

Waluigi Effect

Goodhart's Law

Value learning

Distribution Shift

Instrumental Convergence

AI Alignment Externals

Preference Optimization

AI Safety Institute

Machine Unlearning

AI Power seeking

What is AI alignment

What is it to solve the alignment problem? — LessWrong

People often talk about “solving the alignment problem.” But what is it to do such a thing? I wrote up some rough notes.

What is it to solve the alignment problem? — LessWrong

https://www.lesswrong.com/posts/AFdvSBNgN2EkAsZZA/what-is-it-to-solve-the-alignment-problem-1

What is it to solve the alignment problem? — LessWrong

AI Control names

AI Oversight + Control — ML Alignment & Theory Scholars

As model develop potential dangerous behaviors, can we develop and evaluate methods to monitor and regulate AI systems, ensuring they adhere to desired behaviors while minimally undermining their efficiency or performance?

AI Oversight + Control — ML Alignment & Theory Scholars

https://www.matsprogram.org/oversight

AI Oversight + Control — ML Alignment & Theory Scholars

Recommendations

/////