Posts
Lazypredict: The automl library
Introduction:
Recently I have started my journey with automl, and explored the sberbank's light automl framework as well auto-eda with pandas profiling in this post. In this post, I will explore the lazypredict framework written by Shankar Pandala sir. In this post, we will first show how to use the library, what are the outputs we get from this, and then finally, we will go in-depth of the code; to see how lazypredict does what it does.
Usage:
For this part, we will just use the github repo's code example. There are two classes, LazyClassifier and LazyRegressor, respectively for classifier and regressor. We can import the classifier class if your problem is classification, and import regressor if you have a regression problem.
X_train, X_test, y_train, y_test = train_test_split(X, y,test_size=.5,random_state =123) clf = LazyClassifier(verbose=0,ignore_warnings=True, custom_metric=None) models,predictions = clf.fit(X_train, X_test, y_train, y_test)
Now, look at the above code. This is the example code for using the LazyClassifier. The fit function takes X_train, X_test, y_train and y_test which are basically training and test dataset to train and validate on.
The models is an aggregate result of all the training, which lists down results for all the models trained. Look at the example result for the boston dataset trained on the lazyclassifier.
print(models) | Model | Accuracy | Balanced Accuracy | ROC AUC | F1 Score | Time Taken | |:-------------------------------|-----------:|--------------------:|----------:|-----------:|-------------:| | LinearSVC | 0.989474 | 0.987544 | 0.987544 | 0.989462 | 0.0150008 | | SGDClassifier | 0.989474 | 0.987544 | 0.987544 | 0.989462 | 0.0109992 | | MLPClassifier | 0.985965 | 0.986904 | 0.986904 | 0.985994 | 0.426 | | Perceptron | 0.985965 | 0.984797 | 0.984797 | 0.985965 | 0.0120046 | | LogisticRegression | 0.985965 | 0.98269 | 0.98269 | 0.985934 | 0.0200036 | | LogisticRegressionCV | 0.985965 | 0.98269 | 0.98269 | 0.985934 | 0.262997 | | SVC | 0.982456 | 0.979942 | 0.979942 | 0.982437 | 0.0140011 | | CalibratedClassifierCV | 0.982456 | 0.975728 | 0.975728 | 0.982357 | 0.0350015 | | PassiveAggressiveClassifier | 0.975439 | 0.974448 | 0.974448 | 0.975464 | 0.0130005 | | LabelPropagation | 0.975439 | 0.974448 | 0.974448 | 0.975464 | 0.0429988 | | LabelSpreading | 0.975439 | 0.974448 | 0.974448 | 0.975464 | 0.0310006 | | RandomForestClassifier | 0.97193 | 0.969594 | 0.969594 | 0.97193 | 0.033 | | GradientBoostingClassifier | 0.97193 | 0.967486 | 0.967486 | 0.971869 | 0.166998 | | QuadraticDiscriminantAnalysis | 0.964912 | 0.966206 | 0.966206 | 0.965052 | 0.0119994 | | HistGradientBoostingClassifier | 0.968421 | 0.964739 | 0.964739 | 0.968387 | 0.682003 | | RidgeClassifierCV | 0.97193 | 0.963272 | 0.963272 | 0.971736 | 0.0130029 | | RidgeClassifier | 0.968421 | 0.960525 | 0.960525 | 0.968242 | 0.0119977 | | AdaBoostClassifier | 0.961404 | 0.959245 | 0.959245 | 0.961444 | 0.204998 | | ExtraTreesClassifier | 0.961404 | 0.957138 | 0.957138 | 0.961362 | 0.0270066 | | KNeighborsClassifier | 0.961404 | 0.95503 | 0.95503 | 0.961276 | 0.0560005 | | BaggingClassifier | 0.947368 | 0.954577 | 0.954577 | 0.947882 | 0.0559971 | | BernoulliNB | 0.950877 | 0.951003 | 0.951003 | 0.951072 | 0.0169988 | | LinearDiscriminantAnalysis | 0.961404 | 0.950816 | 0.950816 | 0.961089 | 0.0199995 | | GaussianNB | 0.954386 | 0.949536 | 0.949536 | 0.954337 | 0.0139935 | | NuSVC | 0.954386 | 0.943215 | 0.943215 | 0.954014 | 0.019989 | | DecisionTreeClassifier | 0.936842 | 0.933693 | 0.933693 | 0.936971 | 0.0170023 | | NearestCentroid | 0.947368 | 0.933506 | 0.933506 | 0.946801 | 0.0160074 | | ExtraTreeClassifier | 0.922807 | 0.912168 | 0.912168 | 0.922462 | 0.0109999 | | CheckingClassifier | 0.361404 | 0.5 | 0.5 | 0.191879 | 0.0170043 | | DummyClassifier | 0.512281 | 0.489598 | 0.489598 | 0.518924 | 0.0119965 |
Now, for doing the same thing with regression, the following code is needed.
reg = LazyRegressor(verbose=0, ignore_warnings=False, custom_metric=None) models, predictions = reg.fit(X_train, X_test, y_train, y_test) print(models)
As you can see, the same type of code is used in both regression and classification. We will now dive into the supervised.py script from which we import the two classes. We will analyze the codes and show how the library works. This part is not necessary for using the library; but it is important if you want to contribute or want to understand what you are using.
Analysis of lazypredict code:
The main file to read is the Supervised.py file. In this file, there are two main classes created. These classes are LazyRegressor and LazyClassifier. In each of the class, there is only main function as of now. We are going to launch a few more functions and attributes in the next launch; but we won't discuss the dev branch performance here.
The only dominant function is fit. Inside this, first we initiate the metrics to store in these lines:
Accuracy = []
B_Accuracy = []
ROC_AUC = []
F1 = []
names = []
TIME = []
predictions = {}
After this, we select the features by data type and create two lists, one for numeric features and another for categorical features. The respective lines are:
numeric_features = X_train.select_dtypes(
include=['int64', 'float64', 'int32', 'float32']).columns
categorical_features = X_train.select_dtypes(
include=['object']).columns
Now, after this, the preprocessors are called and we pre-process the categorical columns using one-hot encoding and the numerics are subjected to scaling. Also, missing value imputation is done in this step.
if type(X_train) is np.ndarray:
X_train = pd.DataFrame(X_train)
X_test = pd.DataFrame(X_test)
numeric_features = X_train.select_dtypes(
include=['int64', 'float64', 'int32', 'float32']).columns
categorical_features = X_train.select_dtypes(
include=['object']).columns
preprocessor = ColumnTransformer(
transformers=[
('numeric', numeric_transformer, numeric_features),
('categorical', categorical_transformer, categorical_features)
])
After doing this, a loop starts with the class of classifiers to be fitted. In each loop the models are fitted, then the f1_score, accuracy, B_accuracy ( balanced accuracy) and custom metrics ( if supplied) are calculated.
for name, model in tqdm(CLASSIFIERS):
start = time.time()
try:
if 'random_state' in model().get_params().keys():
pipe = Pipeline(steps=[
('preprocessor', preprocessor),
('classifier', model(random_state = self.random_state))
])
else:
pipe = Pipeline(steps=[
('preprocessor', preprocessor),
('classifier', model())
])
pipe.fit(X_train, y_train)
y_pred = pipe.predict(X_test)
accuracy = accuracy_score(y_test, y_pred, normalize=True)
b_accuracy = balanced_accuracy_score(y_test, y_pred)
f1 = f1_score(y_test, y_pred, average='weighted')
try:
roc_auc = roc_auc_score(y_test, y_pred)
except Exception as exception:
roc_auc = None
if self.ignore_warnings == False:
print("ROC AUC couldn't be calculated for "+name)
print(exception)
If one chooses the verbose parameter as something more than 0, then every model is printed after it is trained.
if self.custom_metric != None:
custom_metric = self.custom_metric(y_test, y_pred)
CUSTOM_METRIC.append(custom_metric)
if self.verbose > 0:
if self.custom_metric != None:
print({"Model": name,
"Accuracy": accuracy,
"Balanced Accuracy": b_accuracy,
"ROC AUC": roc_auc,
"F1 Score": f1,
self.custom_metric.__name__: custom_metric,
"Time taken": time.time() - start})
else:
print({"Model": name,
"Accuracy": accuracy,
"Balanced Accuracy": b_accuracy,
"ROC AUC": roc_auc,
"F1 Score": f1,
"Time taken": time.time() - start})
This goes in the loop. The predictions, accuracies in different metrics are calculated and finally it is returned according to the initialized parameters; i.e. if predictions is set to true, the predictions from the different models are printed; as well as the models data is printed. We have already seen how the models object prints out all the performance of trained models.
if self.predictions == True:
predictions_df = pd.DataFrame.from_dict(predictions)
return scores, predictions_df if self.predictions == True else scores
There are a few checks as the loop runs inside a try except loop, which escapes the process with exception whenever a model fails to train.
As the two classes ( lazyregressor and lazyclassifier) have absolutely same structure other than the list of models they call; so here ends our explanation of how lazypredict works. I will encourage you to take more detailed look into the file on your own if you are interested; or want to contribute to this up and coming library.
Issues and development efforts:
Lazypredict is a fairly new library; hence it is still developing. A few upcoming improvements in the next release contain a provide_models method for both classes which returns the model objects trained, extended dependency list to make using the library more smooth and dropping some deprecated features. The developers behind lazypredict are also discussing features like time out, specific categorical treatment as wanted by the user (rather than the current default one-hot encoding). There are a lot of improvements to come in future versions, and if I were you, I would keep a eye out for this library in future.
Conclusion:
So in this post, not only did we chalk out how to use lazypredict library for your work and automl scripting; but also we took a deep dive inside the code and discussed how it does what it does. If you have read it upto this point; pat your back and thanks for reading! stay tuned and subscribe to this blog to get this and more such awesome machine learning posts every week.
References:
(1) kaggle notebook
Comments
Post a Comment