Stat 479: Machine Learning Lecture Notes

Document Outline

• Nearest Neighbor Methods
  • Introduction
    • Key concepts
    • Nearest Neighbor Classification In Context
    • Common Use Cases of kNN
  • Nearest Neighbor Algorithm
  • Nearest Neighbor Decision Boundary
  • k-Nearest Neighbor Classification and Regression
    • Classification
    • Regression
  • Curse of Dimensionality
  • Computational Complexity and the Big-O Notation
    • Big O of kNN
  • Improving Computational Performance
    • Naive kNN Algorithm in Pseudocode
    • Data Structures
    • Dimensionality Reduction
    • Faster Distance Metric/Heuristic
    • ``Pruning''
    • Parallelizing kNN
  • Distance measures
    • Discrete Features
    • Feature Weighting
  • Distance-weighted kNN
  • Improving Predictive Performance
  • Error Bounds
  • kNN from a Bayesian Perspective
  • Advantages and Disadvantages of kNN
  • Other Forms of Instance-based Learning
    • Locally Weighted Regression
    • Kernel Methods
  • kNN in Python
  • Resources
  • Assigned Reading
  • Further Reading