3 Important Types of Vector Norm Used in Machine Learning

# 3 Important Types of Vector Norm Used in Machine Learning

Vikram Singh
Assistant Manager - Content
Updated on Feb 3, 2023 16:28 IST

The length or the magnitude of the vector is known as vector norm or vector magnitude. In mathematics, a function is defined on a vector space that maps each vector to its magnitude (a scalar quantity). In this article, we will discuss different types of vector norm that are used in Machine Learning.

In the previous articles, we discussed how vectors are used in machine learning. In this article, we will discuss the norm of a vector. A norm is a way to measure the size of a vector, matrix, or tensor; it helps to quantify the magnitude of a vector. In simple terms, the length of the vector is referred to as the vector norm or the vector’s magnitude.

In machine learning, vector norms are used to evaluate the model’s error or reduce the model’s overall complexity.

This article will discuss what vector norm is, the types of vector norms, and how to calculate the vector norm using python.

Let’s explore the article.

## What is Vector Norm

The length or the magnitude of the vector is known as vector norm or vector magnitude. In mathematics, a function is defined on a vector space that maps each vector to its magnitude (a scalar quantity).
In Machine learning, we use them to measure the distance between multiple data points in different dimensions.

For example, the euclidean distance formula is the most often used formula to calculate the distance between any two points.

Let there be two points in 2-dimensional vector space:

P = (p1, p2)

Q = (q1, q2).

Then the distance between P and Q can be calculated by:

dist(P, Q) = [(p1 – q1)2 + (q1 – q2)2](½)

Example: Find the distance between P (2, 3) and Q (4, 5).

Answer: The distance between P and Q is:
dist(P, Q) = [(4 – 2)2 + (5 – 3)2]½
=> dist(P, Q) = [22 + 22]½
=> dist(P, Q) = [4 + 4]½
Hence, dist(P, Q) = 2.83 units

#### Notation

Let x be any vector, then the norm of x is given by: ||x||.

### Properties of Vector Norm

• Positive Definiteness: The norm (or the magnitude) of a vector is always positive.
• The norm value is zero if and only if the vector is zero.
• i.e., ||x||>= 0, for all non-zero vector x.
• Triangle Inequality: The sum of the norm of two vectors is always less than the sum of their (individual) norm.
• If x and y are two vectors, then,
• ||x + y|| <= ||x|| + ||y||
• Homogeneity: Let k be any constant, and x be any vector then:
• ||kx|| = |k|||x||

### Application of Norms of Vector in Machine Learning

• Used to evaluate the error in the model, i.e., the distance between the observed value and predicted value.
• Vector norms are used in regularization, which is a technique to prevent overfitting in the model.

Now, we will see the different types of Vector Norms.

## Vector Norms in Machine Learning

There are different types of vector norms that can be used, such as L1, L2, Lp, L-infinity, Min-Max norm, etc.
But in machine learning, we mainly use L1, L2, and Min-Max Norm.
So, without further delay, let’s start with Lp Norm as all are special cases of Lp norm.

### Lp Norm

It is defined as the absolute sum of the absolute value of the elements of the vector.

Note: L^p norm is the generalization of Euclidean Distance.

It is defined as the sum of the absolute value of the elements of the vector, raised to the pth power and then taking the pth root of the result. The Lp norm is written as ||x||p, where x is the vector and p is a positive real number.

#### Python Program to Calculate the Lp Norm:

` `
`#Import norm from linalg (linear algebra) module of NumPy Libraryimport numpy as npfrom numpy.linalg import norm #define a vector using numpy array v = np.array([1, 4, 9, 16, 25]) #find the L-p norm for p = 3p = 3vector_norm = norm(v, ord = p)print('The vector norm of v is:', vector_norm)Copy code`

Output

The vector norm of v is: 27.375191994928183

#### L1-Norm

If p = 1, then the Lp norm is known as L1 – Norm. It is defined as the sum of the absolute value of the components of a given vector.

Let x be any vector, then the L1 norm of a vector is given by:

• L1-Norm is also known as Manhattan or the Taxicab Norm.
• It is typically the distance a taxi will have to cover from the origin.
• It is used in LASSO Regression.

#### L2-Norm

If p = 2, then the Lp norm is known as L2 – Norm. It is defined as the square root of the sum of the squared vector values.

Let x be any vector, then the L2 norm of a vector is given by:

Must Read: Ridge Regression vs Lasso Regression

#### L infinity-Norm

If p = infinity, then the Lp norm is known as the L infinity norm. It is defined as the absolute value of the largest component of the vector.

It is also called the Max Norm.

## Conclusion

• What is Vector Norm
• Properties and Application of Vector Norm
• How to Calculate the Norm of a vector
• Lp Norm

Top Trending Article

Interview Questions