Software engineering

from sklearn.ensemble import GradientBoostingClassifier model = GradientBoostingClassifier() model.fit(X, Y)
Regression
from sklearn.ensemble import GradientBoostingRegressor model = GradientBoostingRegressor() model.fit(X, Y)
Advantages and disadvantages
In terms of advantages, gradient boosting method is robust to missing data, highly correlated features, and irrelevant features in the same way as random forest. It naturally assigns feature
importance scores, with slightly better performance than random forest. The algorithm doesn’t
need the data to be scaled and can model a nonlinear relationship.
In terms of disadvantages, it may be more prone to overfitting than random forest, as the main purpose of the boosting approach is to reduce bias and not variance. It has many hyperparameters to tune, so model development may not be as fast. Also, feature importance may not be robust to variation in the training dataset.
ANN-Based Models
In Chapter 3 we covered the basics of ANNs, along with the architecture of ANNs and their training and implementation in Python. The details provided in that chapter are applicable across all areas of machine learning, including supervised learning. However, there are a few additional details from the supervised learning perspective, which we will cover in this section.
Neural networks are reducible to a classification or regression model with the activation function of the node in the output layer. In the case of a regression problem, the output node has linear activation function (or no activation function). A linear function produces a continuous output ranging from -inf to +inf. Hence, the output layer will be the linear function of the nodes in the layer before the output layer, and it will be a regression-based model.
In the case of a classification problem, the output node has a sigmoid or softmax activation function. A sigmoid or softmax function produces an output ranging from zero to one to represent the probability of target value. Softmax function can also be used for multiple groups for classification.
ANN using sklearn
ANN regression and classification models can be constructed using the sklearn package of Python, as shown in the following code snippet:
Classification