Bagging

Bootstrap AGGregation

Bagging is a special case of the model averaging approach

Bagging (Bootstrap AGGregation) is a powerful ensemble learning method that creates multiple models trained on randomly sampled subsets of the original dataset. This technique effectively reduces variance and helps prevent overfitting.

Key Characteristics

Creates multiple training datasets through random sampling with replacement (bootstrapping)

Commonly used with decision trees, though applicable to various model types

Functions as a specialized form of model averaging

Implementation Methods

Random Patches: Combines both feature and dataset sampling (bootstrapping)

Random Subspace: Utilizes only feature sampling

Statistical Impact

Feature bagging typically increases bias while reducing variance, whereas dataset bagging maintains similar bias levels while decreasing variance. When compared to single-model training on the complete dataset, bagging generally maintains similar bias but achieves lower variance through the use of mode (most frequent value) for final predictions.

In statistical terms, bootstrapping refers to resampling with replacement, allowing the same data point to be selected multiple times during the sampling process.

[머신러닝] 앙상블 학습 - 2) Bagging

지난 번 포스팅에서 앙상블 학습의 의미와 효과에 대해서 알아보았다. 이어서 이번 포스팅에서는 앙상블 학습 기법 중 한 종류인 Bagging에 대해서 정리하였다. 구체적으로는, 1) Bagging의 정의와 의의, 그리고 2) 모델 결합 방법에 대해서 알아본다. 본 포스팅은 고려대학교 강필성 교수님의 비즈니스 애널리틱스 강의 내용을 요약한 것이며, 해당 강의는 유튜브에서 무료로 시청이 가능하다(링크). 아래 링크들은 본 포스팅과 관련된 게시글이므로 같이 보면 이해가 더 쉬울 것이다. 1. [머신러닝] 편향-분산 분해 2. [머신러닝] 편향과 분산의 의미 3. [머신러닝] 앙상블 학습 - 1) 배경 1. Bagging 1) Bagging (Bootstrap Aggregating) 이란? Bagging이란, 기..

https://sungkee-book.tistory.com/9

Bagging

Bootstrap AGGregation

Key Characteristics

Implementation Methods

Statistical Impact

Backlinks

Recommendations