05 - K-Nearest Neighbor - lec. 6,7

ucla | CS M146 | 2023-04-24T13:58

---

Table of Contents

• Supplemental
• Lecture
  • 1-Nearest Neighbor (NN) Classification
    • Data Visualization (Flower types)
    • 1-Nearest Neighbor
    • Calculating Distances
    • Decision Boundaries
  • K-Nearest Neighbor (KNN) Classification
    • Visual
    • Voronoi diagrams for KNN
• Discussion
• Resources

Supplemental

Lecture

1-Nearest Neighbor (NN) Classification

Data Visualization (Flower types)

1-Nearest Neighbor

• label a data point to the class of the data point closest to it

Calculating Distances

• L2 norm is Euclidean distance

• L1 distance

Decision Boundaries

• nearly all boundaries so far have been linear
• nearest neighbor can have a non-linear decision boundary

K-Nearest Neighbor (KNN) Classification

• Get indices of the top $k$ nearesst neighbors

• do majority vote to classify data points

• KNN assumes all features are equally useful for classification
• scale of measurements matters → there will be different nighbors if features are not on the same scale of magnitude → we van usescale to give some features more weight
• treat $k$ as a hyperparam to optimize

Visual

Voronoi diagrams for KNN

Discussion

Resources

---

📌

**SUMMARY
**