How to Build an End-to-End Production Grade Machine Learning Pipeline with ZenML, Including Custom Materializers, Metadata Tracking, and Hyperparameter Optimization
@step(enable_cache=True) def load_data() -> Annotated[DatasetBundle, “raw_dataset”]: data = load_breast_cancer() return DatasetBundle( data.data, data.target, data.feature_names, stats={“source”: “sklearn.datasets.load_breast_cancer”}, ) @step def split_and_scale( bundle: DatasetBundle, test_size: float = 0.2, random_state: int = 42, ) -> Tuple[ Annotated[np.ndarray, “X_train”], Annotated[np.ndarray, “X_test”], Annotated[np.ndarray, “y_train”], Annotated[np.ndarray, “y_test”], ]: X_tr, X_te, y_tr, y_te = train_test_split( bundle.X, bundle.y, test_size=test_size, random_state=random_state, stratify=bundle.y, ) scaler…