All that You Need to Know About Sigmoid Function

# All that You Need to Know About Sigmoid Function

Vikram Singh
Assistant Manager - Content
Updated on Oct 12, 2023 15:42 IST

The sigmoid function is a special case of a logistic function that has S-shaped characteristic and are used as an activation function in Neural Networks. In this article, we will briefly discuss logistic function, sigmoid function.

As in linear regression algorithms, we use linear function or equation to give the relationship between dependent and independent variables; similarly, in logistic regression algorithms, we use logistic function (or sigmoid function). A logistic function is an exponential function that has a characteristic S-shaped curve.

In logistic regression, we mainly use the sigmoid function, a particular Logistic Function case. This article will briefly discuss the logistic function and the Sigmoid Function.
Now, let’s dive deep to learn more about Logistic and sigmoid functions and how to use them.

## What is Logistic Function?

The logistic function is a particular case of the exponential function that is used to model the exponential growth of a population that is derived from the differential equation:

Now, to solve the above differential equation, substitute:

in the above equation, we get the following:

and on solving the above differential equation, we will get the following:

where:

x0: is the initial value at time t0

The above equation (eqn. – (iii)), is known as Logistic Function.

## What is Sigmoid Function?

The sigmoid function is a particular case of logistic function (eqn. – (iii)) when the initial time (t0 = 0), x0 = ½, and r = 1.

So, substituting the above values (t0 = 0, and x0 = ½) in equation (iii), we will get:

The sigmoid function looks like this:

From, the above figure you can easily depict why sigmoid functions are also called S-shaped curve.

• The sigmoid function is the solution of the non-linear differential equation:
• Mainly used with a non-linear activation function.
• It is used with the model where we have to predict the probability as an output of the model.
• Since the probability value lies between 0 and 1.

### Properties of Sigmoid Function

• Domain: (-infinite, +infinite)
• Range: (0, 1)
• x(0) = ½ = 0.5
• The sigmoid function is continuous and monotonically increasing everywhere.
• The function is differentiable everywhere.
• Maps feature space into probability function:
• When x-> + infinity, the value of the sigmoid function will be close to 1.
• When x-> -infinity, the value of the sigmoid function will be close to 0.
• For x = 0, the value of the sigmoid function will be ½.

### Application of Sigmoid Function

• Used as an activation function in Neural Networks.
• Used to predict the probability of binary variables.
• It is used in the Logistic Regression Algorithm.

Also Read: ReLU and Sigmoid Activation Function

### Advantages and Limitations of Sigmoid Function

• It gives a smooth gradient that prevents jumps in output values.
• One of the best-normalized functions.
• When used with a linear function, it will return a value between 0 and 1, which does not make the activation value disappear.
• Gradient values are only significant for the range -3 to 3.
• The graph will have minimal gradients for values greater than 3 or smaller than -3.
• The main disadvantage of the sigmoid function is that it suffers from the vanishing gradient problem.
• As the gradient value approaches zero, the network ceases to learn and suffers from a vanishing gradient descent problem.
• When sigmoid is used and:
• Inputs are non-zero centered; then saturation is a concern
• Saturation implies gradient will be zero.
• Extremely greater or smaller values are mapped to extremities to 0 and 1.

## Implementation of Sigmoid Function in Python

Now, we will create the graph of sigmoid function in Python:

` `
```#import important libraries import numpy as npimport mathfrom matplotlib import pyplot as plt #use numpy.linspace (start, stop, num) that will return number spaces with respect to interval# num: Number of samples to generate x = np.linspace(-20, 20, 500) #define sigmoid functiony = 1/ (1 + np.exp(-x)) #plot the function plt.plot(x, y)plt.xlabel("X")plt.ylabel("Sigmoid (X)") plt.show()Copy code```

Output

## Conclusion

This article briefly discusses logistic function, sigmoid function, properties, application, and implementation of sigmoid function in python.

Hope you will like the article.

Top Trending Article

Interview Questions

## FAQs

What is a Sigmoid Function?

A sigmoid function is a mathematical function that produces a characteristic S shaped curve or sigmoid curve. In data science, it's often used in logistic regression and artificial neural networks to map predicted values to probabilities.

Why are Sigmoid Functions Important in Machine Learning?

Sigmoid functions are vital in machine learning as they help in predicting the probability of an outcome in a binary classification problem, which can be essential in decision-making processes.

How Does a Sigmoid Function Work?

The sigmoid function takes any range of input and outputs it between 0 and 1, making it particularly useful in algorithms where the output needs to be a probability.

Can Sigmoid Functions be used in Multi-class Classification?

While sigmoid functions are primarily used for binary classification, they can be extended to multi-class classification problems using techniques like One-Vs-Rest.

What are the Limitations of Sigmoid Functions?

Sigmoid functions can sometimes suffer from the vanishing gradient problem, where the gradients become too small for the network to learn effectively. This can slow down the training process and make the model less accurate.