05 - Bias-Variance Tradeoff - lec. 6

ucla | CS M146 | 2023-04-19T15:11

---

Table of Contents

• Supplemental
• Lecture
  • Bias-Variance Tradeoff
    • Bias vs. Variance
    • Bias of an estimator
    • Variance
    • Error Decomposition
    • Learning Curves
    • Assessing Error via Learning Curves
• Discussion
• Resources

Supplemental

Lecture

Bias-Variance Tradeoff

Bias vs. Variance

Bias of an estimator

• difference bw an estimator’s expectd value and true value
• ghow far a ML model’s predictions drift from the true predictions
• high bias → underfitting

Variance

• if we train an estimator / ML model on different training sets → how much do predictions vary
• high variance → overfitting

Error Decomposition

• For regression: the expected squared error for any hypo. can be decomposed to
• noise in training data, Bias^2, and Variance

• fixing errors

Learning Curves

• model complexity curves
  • performance (y-axis) as a function of complexity (x-axis)
• learning curves
  • performance (y-axis) as a func. of training size (x-axis)
• magnitude of training error → bias
• gap in training to validation → variance

Assessing Error via Learning Curves

• Identifying features of curves of validation and training

• Ideal Learning curve

Discussion

Resources

---

📌

**SUMMARY
**