Software engineering

from sklearn.linear_model import Lasso model = Lasso () model.fit(X, Y)
L2 regularization or Ridge regression
Ridge regression performs L2 regularization by adding a factor of the sum of the square of coefficients in the cost function (RSS) for linear regression, as mentioned in Equation 4-1. The equation for ridge regularization can be represented as follows:
C o s t F u n c t i o n = R S S + X * £ j=1 p P j 2
Ridge regression puts constraint on the coefficients. The penalty term ( X ) regularizes the coefficients such that if the coefficients take large values, the optimization function is penalized. So ridge regression shrinks the coefficients and helps to reduce the model complexity. Shrinking the coefficients leads to a lower variance and a lower error value. Therefore, ridge regression decreases the complexity of a model but does not reduce the number of variables; it just shrinks their effect. When X is closer to zero, the cost function becomes similar to the linear regression cost function. So the lower the constraint (low X ) on the features, the more the model will resemble the linear regression model.
A ridge regression model can be constructed using the Ridge class of the sklearn package of Python, as shown in the code snippet that follows:
from sklearn.linear_model import Ridge model = Ridge () model.fit(X, Y)
Elastic net
Elastic nets add regularization terms to the model, which are a combination of both L1 and L2 regularization, as shown in the following equation:
C o s t F u n c t i o n = R S S + X * (1-a) / 2 * £ j=1 p P j 2 + a * £ j=1 p P j
In addition to setting and choosing a X value, an elastic net also allows us to tune the alpha parameter, where a = 0 corresponds to ridge and a = 1 to lasso. Therefore, we can choose an a value between 0 and 1 to optimize the elastic net. Effectively, this will shrink some coefficients and set some to 0 for sparse selection.
An elastic net regression model can be constructed using the ElasticNet class of the sklearn package of Python, as shown in the following code snippet: