We start this tutorial to display learn how to harness TPOT to automate and optimize machine studying pipelines virtually. By working straight in Google Colab, we make sure the setup is light-weight, reproducible, and accessible. We stroll by way of loading information, defining a customized scorer, tailoring the search house with superior fashions like XGBoost, and establishing a cross-validation technique. As we proceed, we discover how evolutionary algorithms in TPOT seek for high-performing pipelines, offering us transparency by way of Pareto fronts and checkpoints. Try the FULL CODES here.
!pip -q set up tpot==0.12.2 xgboost==2.0.3 scikit-learn==1.4.2 graphviz==0.20.3
import os, json, math, time, random, numpy as np, pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, StratifiedKFold
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import make_scorer, f1_score, classification_report, confusion_matrix
from sklearn.pipeline import Pipeline
from tpot import TPOTClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.naive_bayes import GaussianNB
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier, ExtraTreesClassifier, GradientBoostingClassifier
from xgboost import XGBClassifier
SEED = 7
random.seed(SEED); np.random.seed(SEED); os.environ["PYTHONHASHSEED"]=str(SEED)We start by putting in the libraries and importing all of the important modules that assist information dealing with, mannequin constructing, and pipeline optimization. We set a hard and fast random seed to make sure our outcomes stay reproducible each time we run the pocket book. Try the FULL CODES here.
X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, stratify=y, random_state=SEED)
scaler = StandardScaler().match(X_tr)
X_tr_s, X_te_s = scaler.remodel(X_tr), scaler.remodel(X_te)
def f1_cost_sensitive(y_true, y_pred):
return f1_score(y_true, y_pred, common="binary", pos_label=1)
cost_f1 = make_scorer(f1_cost_sensitive, greater_is_better=True)Right here, we load the breast most cancers dataset and cut up it into coaching and testing units whereas preserving class steadiness. We standardize the options for stability after which outline a customized F1-based scorer, permitting us to guage pipelines with a concentrate on successfully capturing optimistic instances. Try the FULL CODES here.
tpot_config = {
'sklearn.linear_model.LogisticRegression': {
'C': [0.01, 0.1, 1.0, 10.0],
'penalty': ['l2'], 'solver': ['lbfgs'], 'max_iter': [200]
},
'sklearn.naive_bayes.GaussianNB': {},
'sklearn.tree.DecisionTreeClassifier': {
'criterion': ['gini','entropy'], 'max_depth': [3,5,8,None],
'min_samples_split':[2,5,10], 'min_samples_leaf':[1,2,4]
},
'sklearn.ensemble.RandomForestClassifier': {
'n_estimators':[100,300], 'criterion':['gini','entropy'],
'max_depth':[None,8], 'min_samples_split':[2,5], 'min_samples_leaf':[1,2]
},
'sklearn.ensemble.ExtraTreesClassifier': {
'n_estimators':[200], 'criterion':['gini','entropy'],
'max_depth':[None,8], 'min_samples_split':[2,5], 'min_samples_leaf':[1,2]
},
'sklearn.ensemble.GradientBoostingClassifier': {
'n_estimators':[100,200], 'learning_rate':[0.03,0.1],
'max_depth':[2,3], 'subsample':[0.8,1.0]
},
'xgboost.XGBClassifier': {
'n_estimators':[200,400], 'max_depth':[3,5], 'learning_rate':[0.05,0.1],
'subsample':[0.8,1.0], 'colsample_bytree':[0.8,1.0],
'reg_lambda':[1.0,2.0], 'min_child_weight':[1,3],
'n_jobs':[0], 'tree_method':['hist'], 'eval_metric':['logloss'],
'gamma':[0,1]
}
}
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=SEED)We outline a customized TPOT configuration that mixes linear fashions, tree-based learners, ensembles, and XGBoost, using rigorously chosen hyperparameters. We additionally established a stratified 5-fold cross-validation technique, making certain that each candidate pipeline is examined pretty throughout balanced splits of the dataset. Try the FULL CODES here.
t0 = time.time()
tpot = TPOTClassifier(
generations=5,
population_size=40,
offspring_size=40,
scoring=cost_f1,
cv=cv,
subsample=0.8,
n_jobs=-1,
config_dict=tpot_config,
verbosity=2,
random_state=SEED,
max_time_mins=10,
early_stop=3,
periodic_checkpoint_folder="tpot_ckpt",
warm_start=False
)
tpot.match(X_tr_s, y_tr)
print(f"n⏱️ First search took {time.time()-t0:.1f}s")
def pareto_table(tpot_obj, ok=5):
rows=[]
for ind, meta in tpot_obj.pareto_front_fitted_pipelines_.gadgets():
rows.append({
"pipeline": ind, "cv_score": meta['internal_cv_score'],
"measurement": len(str(meta['pipeline'])),
})
df = pd.DataFrame(rows).sort_values("cv_score", ascending=False).head(ok)
return df.reset_index(drop=True)
pareto_df = pareto_table(tpot, ok=5)
print("nTop Pareto pipelines (cv):n", pareto_df)
def eval_pipeline(pipeline, X_te, y_te, title):
y_hat = pipeline.predict(X_te)
f1 = f1_score(y_te, y_hat)
print(f"n[{name}] F1(take a look at) = {f1:.4f}")
print(classification_report(y_te, y_hat, digits=3))
print("nEvaluating high pipelines on take a look at:")
for i, (ind, meta) in enumerate(sorted(
tpot.pareto_front_fitted_pipelines_.gadgets(),
key=lambda kv: kv[1]['internal_cv_score'], reverse=True)[:3], 1):
eval_pipeline(meta['pipeline'], X_te_s, y_te, title=f"Pareto#{i}")We launch an evolutionary search with TPOT, cap the runtime for practicality, and checkpoint progress, permitting us to reproducibly hunt for robust pipelines. We then examine the Pareto entrance to determine the highest trade-offs, convert it right into a compact desk, and choose leaders based mostly on the cross-validation rating. Lastly, we consider one of the best candidates on the held-out take a look at set to substantiate real-world efficiency with F1 and a full classification report. Try the FULL CODES here.
print("n🔁 Heat-start for additional refinement...")
t1 = time.time()
tpot2 = TPOTClassifier(
generations=3, population_size=40, offspring_size=40,
scoring=cost_f1, cv=cv, subsample=0.8, n_jobs=-1,
config_dict=tpot_config, verbosity=2, random_state=SEED,
warm_start=True, periodic_checkpoint_folder="tpot_ckpt"
)
attempt:
tpot2._population = tpot._population
tpot2._pareto_front = tpot._pareto_front
besides Exception:
cross
tpot2.match(X_tr_s, y_tr)
print(f"⏱️ Heat-start additional search took {time.time()-t1:.1f}s")
best_model = tpot2.fitted_pipeline_ if hasattr(tpot2, "fitted_pipeline_") else tpot.fitted_pipeline_
eval_pipeline(best_model, X_te_s, y_te, title="BestAfterWarmStart")
export_path = "tpot_best_pipeline.py"
(tpot2 if hasattr(tpot2, "fitted_pipeline_") else tpot).export(export_path)
print(f"n📦 Exported finest pipeline to: {export_path}")
from importlib import util as _util
spec = _util.spec_from_file_location("tpot_best", export_path)
tbest = _util.module_from_spec(spec); spec.loader.exec_module(tbest)
reloaded_clf = tbest.exported_pipeline_
pipe = Pipeline([("scaler", scaler), ("model", reloaded_clf)])
pipe.match(X_tr, y_tr)
eval_pipeline(pipe, X_te, y_te, title="ReloadedExportedPipeline")
report = {
"dataset": "sklearn breast_cancer",
"train_size": int(X_tr.form[0]), "test_size": int(X_te.form[0]),
"cv": "StratifiedKFold(5)",
"scorer": "customized F1 (binary)",
"search": {"gen_1": 5, "gen_2_warm": 3, "pop": 40, "subsample": 0.8},
"exported_pipeline_first_120_chars": str(reloaded_clf)[:120]+"...",
}
print("n🧾 Mannequin Card:n", json.dumps(report, indent=2))We proceed the search with a heat begin, reusing the realized heat begin to refine candidates and choose one of the best performer on our take a look at set. We export the successful pipeline, reload it alongside our scaler to imitate deployment, and confirm its outcomes. Lastly, we generate a compact mannequin card to doc the dataset, search settings, and the abstract of the exported pipeline for reproducibility.
In conclusion, we see how TPOT permits us to maneuver past trial-and-error mannequin choice and as a substitute depend on automated, reproducible, and explainable optimization. We export one of the best pipeline, validate it on unseen information, and even reload it for deployment-style use, confirming that the workflow is not only experimental however production-ready. By combining reproducibility, flexibility, and interpretability, we finish with a strong framework that we will confidently apply to extra advanced datasets and real-world issues.
Try the FULL CODES here. Be at liberty to take a look at our GitHub Page for Tutorials, Codes and Notebooks. Additionally, be at liberty to observe us on Twitter and don’t overlook to affix our 100k+ ML SubReddit and Subscribe to our Newsletter.
Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is dedicated to harnessing the potential of Synthetic Intelligence for social good. His most up-to-date endeavor is the launch of an Synthetic Intelligence Media Platform, Marktechpost, which stands out for its in-depth protection of machine studying and deep studying information that’s each technically sound and simply comprehensible by a large viewers. The platform boasts of over 2 million month-to-month views, illustrating its reputation amongst audiences.
