https://bib.dbvis.de/uploadedFiles/155.pdf

University of Michigan - Ann Arbor

The power parameter p controls which distance metric is used in determining neighbors of a point. The default is p=2, which specifies Euclidean distance. Setting p=1 specifies Manhattan distance (aka city block distance), which calculates distance along a grid-like path. Manhattan distance is preferred for high-dimensional data.

"p" Parameter for KNN Classifier

https://medium.com/@kunal_gohrani/different-types-of-distance-metrics-used-in-machine-learning-e9928c5e26c7#:~:text=Manhattan%20distance%20is%20usually%20preferred,similarity%20between%20two%20data%20points.

Different Types of Distance Metrics used in Machine Learning

Manhattan distance, also known as city block distance, is the number of units one must traverse to get from point X to point Y along a grid-like path. Each step involves a change in value across exactly one dimension. In machine learning, Manhattan distance is often preferred over higher values of $$p$$ for high-dimensionality data due to simpler computation and relative robustness against the curse of high dimensionality. More formally, let $$x_i$$ be the value of point X across the $$i$$-th dimension, and let $$y_i$$ be the value of point Y across the $$i$$-th dimension. In $$n$$ dimensions, the Manhattan distance between points X and Y is: $$d(X,Y) = \sum_{i=1}^n|x_i - y_i|$$. This metric is an instantiation of the Minkowski distance formula where $$p=1$$.

Manhattan Distance (p=1)

The curse of high dimensionality is the empirical phenomenon that in high dimensional space, concepts of proximity, distance, and/or nearest neighbor lose qualitative meaning. This problem is more severe for Euclidean distance than for Manhattan distance, hence why Manhattan distance is preferred for high dimensional data.

Curse of High Dimensionality

Euclidean distance is the straight-line distance between two points in a Euclidean space. For two points $$x$$ and $$y$$ in a $$k$$-dimensional space, the distance is calculated as the square root of the sum of the squared differences between their coordinates:

$$d(x, y) = \\sqrt{ \\sum_{i=1}^k (x_i - y_i)^2 }$$

Learn Before

Related