3 Important Types of Vector Norm Used in Machine Learning

3 Important Types of Vector Norm Used in Machine Learning

5 mins read1.3K Views Comment
Vikram
Vikram Singh
Assistant Manager - Content
Updated on Feb 3, 2023 16:28 IST

The length or the magnitude of the vector is known as vector norm or vector magnitude. In mathematics, a function is defined on a vector space that maps each vector to its magnitude (a scalar quantity). In this article, we will discuss different types of vector norm that are used in Machine Learning.

2023_01_MicrosoftTeams-image-125.jpg

In the previous articles, we discussed how vectors are used in machine learning. In this article, we will discuss the norm of a vector. A norm is a way to measure the size of a vector, matrix, or tensor; it helps to quantify the magnitude of a vector. In simple terms, the length of the vector is referred to as the vector norm or the vector’s magnitude.

In machine learning, vector norms are used to evaluate the model’s error or reduce the model’s overall complexity. 

This article will discuss what vector norm is, the types of vector norms, and how to calculate the vector norm using python.

Let’s explore the article.

Must Read: How Much Mathematics do we have to learn for Data Science

Programming Online Courses and Certification Python Online Courses and Certifications
Data Science Online Courses and Certifications Machine Learning Online Courses and Certifications

Table of Content

What is Vector Norm

The length or the magnitude of the vector is known as vector norm or vector magnitude. In mathematics, a function is defined on a vector space that maps each vector to its magnitude (a scalar quantity).
In Machine learning, we use them to measure the distance between multiple data points in different dimensions.

For example, the euclidean distance formula is the most often used formula to calculate the distance between any two points.

Let there be two points in 2-dimensional vector space:

P = (p1, p2)

Q = (q1, q2).

Then the distance between P and Q can be calculated by:

dist(P, Q) = [(p1 – q1)2 + (q1 – q2)2](½)

Example: Find the distance between P (2, 3) and Q (4, 5).

Answer: The distance between P and Q is:
dist(P, Q) = [(4 – 2)2 + (5 – 3)2]½
=> dist(P, Q) = [22 + 22]½
=> dist(P, Q) = [4 + 4]½
Hence, dist(P, Q) = 2.83 units

Notation

Let x be any vector, then the norm of x is given by: ||x||.

What is Programming What is Python
What is Data Science What is Machine Learning

Properties of Vector Norm

  • Positive Definiteness: The norm (or the magnitude) of a vector is always positive.
    • The norm value is zero if and only if the vector is zero.
      • i.e., ||x||>= 0, for all non-zero vector x.
  • Triangle Inequality: The sum of the norm of two vectors is always less than the sum of their (individual) norm.
    • If x and y are two vectors, then,
      • ||x + y|| <= ||x|| + ||y||  
  • Homogeneity: Let k be any constant, and x be any vector then:
    • ||kx|| = |k|||x||
Set Theory Differential Calculus
Maxima and Minima Functions in Mathematics
Mean Value Theorem Taylor Series

Application of Norms of Vector in Machine Learning

  • Used to evaluate the error in the model, i.e., the distance between the observed value and predicted value.
  • Vector norms are used in regularization, which is a technique to prevent overfitting in the model.

Now, we will see the different types of Vector Norms.

Vector Norms in Machine Learning

There are different types of vector norms that can be used, such as L1, L2, Lp, L-infinity, Min-Max norm, etc.
But in machine learning, we mainly use L1, L2, and Min-Max Norm.
So, without further delay, let’s start with Lp Norm as all are special cases of Lp norm.

Types of Matrix All about Symmetric Matrix
Matrix Multiplication Transpose of a Matrix

Lp Norm

It is defined as the absolute sum of the absolute value of the elements of the vector.

2023_01_image-48.jpg

Note: L^p norm is the generalization of Euclidean Distance.

It is defined as the sum of the absolute value of the elements of the vector, raised to the pth power and then taking the pth root of the result. The Lp norm is written as ||x||p, where x is the vector and p is a positive real number. 

Python Program to Calculate the Lp Norm:

 
#Import norm from linalg (linear algebra) module of NumPy Library
import numpy as np
from numpy.linalg import norm
#define a vector using numpy array
v = np.array([1, 4, 9, 16, 25])
#find the L-p norm for p = 3
p = 3
vector_norm = norm(v, ord = p)
print('The vector norm of v is:', vector_norm)
Copy code

Output

The vector norm of v is: 27.375191994928183

L1-Norm

If p = 1, then the Lp norm is known as L1 – Norm. It is defined as the sum of the absolute value of the components of a given vector.

Let x be any vector, then the L1 norm of a vector is given by:

2023_01_image-49.jpg
  • L1-Norm is also known as Manhattan or the Taxicab Norm.
    • It is typically the distance a taxi will have to cover from the origin.
    • It is used in LASSO Regression.

L2-Norm

If p = 2, then the Lp norm is known as L2 – Norm. It is defined as the square root of the sum of the squared vector values.

Let x be any vector, then the L2 norm of a vector is given by:

2023_01_image-50.jpg

Must Read: Ridge Regression vs Lasso Regression

L infinity-Norm

If p = infinity, then the Lp norm is known as the L infinity norm. It is defined as the absolute value of the largest component of the vector.

2023_01_image-51.jpg

It is also called the Max Norm.

Conclusion

In this article, we have learned the following:

  • What is Vector Norm
  • Properties and Application of Vector Norm
  • How to Calculate the Norm of a vector
  • Lp Norm
Cross Product of Two Vectors Difference between Eigen Value and Eigen Vector

Top Trending Article

Top Online Python Compiler | How to Check if a Python String is Palindrome | Feature Selection Technique | Conditional Statement in Python | How to Find Armstrong Number in Python | Data Types in Python | How to Find Second Occurrence of Sub-String in Python String | For Loop in Python |Prime Number | Inheritance in Python | Validating Password using Python Regex | Python List |Market Basket Analysis in Python | Python Dictionary | Python While Loop | Python Split Function | Rock Paper Scissor Game in Python | Python String | How to Generate Random Number in Python | Python Program to Check Leap Year | Slicing in Python

Interview Questions

Data Science Interview Questions | Machine Learning Interview Questions | Statistics Interview Question | Coding Interview Questions | SQL Interview Questions | SQL Query Interview Questions | Data Engineering Interview Questions | Data Structure Interview Questions | Database Interview Questions | Data Modeling Interview Questions | Deep Learning Interview Questions |

About the Author
author-image
Vikram Singh
Assistant Manager - Content

Vikram has a Postgraduate degree in Applied Mathematics, with a keen interest in Data Science and Machine Learning. He has experience of 2+ years in content creation in Mathematics, Statistics, Data Science, and Mac... Read Full Bio