{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# BitBullet:Lessons: Regression\n", "\n", "Regression is the supervised learning task where the target is continuous: demand, price, revenue, risk exposure, delivery time, or any numeric outcome where the size of the error matters.\n", "\n", "In this lesson we build a complete regression workflow with the BitBullet SDK: feature profiling, leakage-safe transformations, baseline training, hyperparameter search, residual analysis, model benchmarking, and model packaging with the metadata needed to replay inference correctly.\n", "\n", "**Dataset:** Bike Sharing Dataset \n", "**Source:** [UCI Machine Learning Repository - Bike Sharing Dataset](https://archive.ics.uci.edu/dataset/275/bike+sharing+dataset) \n", "**File:** `hour.csv` \n", "**Task:** Predict the hourly number of rented bikes (`cnt`) from calendar, season, weather, and environmental features.\n", "\n", "> Before running this notebook, download the Bike Sharing Dataset and place `hour.csv` next to this notebook, or update `data_path` in Section 2.\n", "\n", "---\n", "\n", "### What You Will Build\n", "\n", "| Step | Component | What BitBullet Handles For You |\n", "|------|-----------|--------------------------------|\n", "| 1 | Feature profiling | `generate_feature_stats` for mixed numeric/categorical audit |\n", "| 2 | Time-aware hold-out split | Train on earlier rows, test on later rows |\n", "| 3 | Transform pipeline | Fit on train only, save, reload, and replay |\n", "| 4 | Baseline model | Fixed-parameter Ridge regression through `TrainConfig` |\n", "| 5 | Optimized model | LightGBM regression with Optuna and TimeSeriesSplit CV |\n", "| 6 | Regression evaluation | `evaluate_regression` metrics, summaries, residuals, and predictions |\n", "| 7 | Benchmarking | Compare model families using the same training interface |\n", "| 8 | Model packaging | `ModelMetadata` and `ModelSerializer` for reproducible artifacts |\n", "| 9 | Inference replay | Load pipeline + model, preserve feature order, predict new rows |\n" ] }, { "cell_type": "markdown", "id": "ea344f8c", "metadata": {}, "source": [ "> **Configure the regression lifecycle without assembling it in code.**\n", "> [BitBullet Platform](https://bitbullet.co.uk/platform/regression) centralises your data, managed compute, storage, modelling configurations, and results in one guided environment. Configure features, transformations, validation, model search, and candidate experiments with clicks or AI-assisted draft preparation; compare fit and residual diagnostics, then export the fitted artefacts, preprocessing, metadata, and generated inference code. This lesson gives you direct SDK control over that workflow." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. Environment and Imports" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import sys\n", "import os\n", "import warnings\n", "import time\n", "from datetime import datetime\n", "from pathlib import Path\n", "\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "\n", "warnings.filterwarnings('ignore')\n", "\n", "# When running this notebook from bitbullet/lessons in a local clone, prefer the local SDK source.\n", "sdk_root = os.path.abspath('..')\n", "if sdk_root not in sys.path:\n", " sys.path.insert(0, sdk_root)\n", "\n", "import bitbullet\n", "print(f\"bitbullet : v{getattr(bitbullet, '__version__', 'dev')}\")\n", "\n", "from bitbullet.transform import TransformPipeline, generate_feature_stats\n", "from bitbullet.train import TrainConfig, OptunaTrainer\n", "from bitbullet.evaluate import evaluate_regression\n", "from bitbullet.model import ModelMetadata, ModelSerializer\n", "\n", "plt.rcParams.update({\n", " 'figure.facecolor': 'white',\n", " 'axes.spines.top': False,\n", " 'axes.spines.right': False,\n", " 'axes.grid': True,\n", " 'grid.alpha': 0.25,\n", " 'font.size': 11,\n", "})\n", "\n", "print('All imports successful.')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. Load the Dataset\n", "\n", "Each row is one hour from the Capital Bikeshare system in Washington, D.C. The target `cnt` is the total number of rented bikes in that hour.\n", "\n", "Two columns are leakage columns: `casual` and `registered`. They sum directly to `cnt`, so they must never be used as model inputs. We drop them before training." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Update this path if you stored the file in a different location.\n", "data_path = 'hour.csv'\n", "\n", "df_raw = pd.read_csv(data_path, parse_dates=['dteday'])\n", "df_raw = df_raw.sort_values('instant').reset_index(drop=True)\n", "\n", "print(f\"Dataset loaded - shape: {df_raw.shape}\")\n", "print(f\"Date range: {df_raw['dteday'].min().date()} to {df_raw['dteday'].max().date()}\")\n", "display(df_raw.head())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. Prepare Features\n", "\n", "The original dataset uses compact integer codes for season, weekday, hour, weather situation, and binary calendar flags. We convert these to categorical strings so the pipeline can encode them deliberately.\n", "\n", "We keep `dteday` only for ordering and explanation. The model already receives month, hour, weekday, and year indicators." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "df = df_raw.copy()\n", "\n", "season_map = {1: 'spring', 2: 'summer', 3: 'fall', 4: 'winter'}\n", "weather_map = {\n", " 1: 'clear_or_partly_cloudy',\n", " 2: 'mist_or_cloudy',\n", " 3: 'light_rain_or_snow',\n", " 4: 'heavy_rain_or_snow',\n", "}\n", "weekday_map = {\n", " 0: 'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday',\n", " 4: 'thursday', 5: 'friday', 6: 'saturday',\n", "}\n", "\n", "df['season'] = df['season'].map(season_map)\n", "df['weathersit'] = df['weathersit'].map(weather_map)\n", "df['weekday'] = df['weekday'].map(weekday_map)\n", "df['yr'] = df['yr'].map({0: '2011', 1: '2012'})\n", "\n", "for col in ['mnth', 'hr', 'holiday', 'workingday']:\n", " df[col] = df[col].astype(str)\n", "\n", "TARGET = 'cnt'\n", "leakage_cols = ['casual', 'registered']\n", "drop_cols = ['instant', 'dteday', TARGET] + leakage_cols\n", "\n", "X = df.drop(columns=drop_cols)\n", "y = df[TARGET]\n", "\n", "print(f\"Features: {X.shape[1]}\")\n", "print(f\"Target range: {y.min()} to {y.max()} rentals per hour\")\n", "display(X.head())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. Exploratory Feature Analysis\n", "\n", "`generate_feature_stats` gives us a compact audit of missingness, skew, kurtosis, cardinality, and feature type. For regression, also inspect the target distribution because error metrics are shaped by the scale and skew of the target." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "stats_df, numerical_cols, categorical_cols = generate_feature_stats(X)\n", "\n", "print(f\"Numerical features ({len(numerical_cols)}): {numerical_cols}\")\n", "print(f\"Categorical features ({len(categorical_cols)}): {categorical_cols}\")\n", "print()\n", "display(stats_df)\n", "\n", "print('\\nTarget summary:')\n", "display(y.describe(percentiles=[0.05, 0.25, 0.5, 0.75, 0.95]))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fig, axes = plt.subplots(1, 2, figsize=(13, 4))\n", "\n", "sns.histplot(y, bins=50, kde=True, ax=axes[0], color='#2563EB')\n", "axes[0].set_title('Hourly rental count distribution')\n", "axes[0].set_xlabel('Rentals per hour')\n", "\n", "hourly_profile = df_raw.groupby('hr')['cnt'].mean()\n", "hourly_profile.plot(ax=axes[1], color='#10B981', marker='o')\n", "axes[1].set_title('Average rentals by hour')\n", "axes[1].set_xlabel('Hour of day')\n", "axes[1].set_ylabel('Average rentals')\n", "\n", "plt.tight_layout()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. Time-Aware Train/Test Split\n", "\n", "This is demand over time, so a random split would make the test set unrealistically similar to the training set. We train on the first 80% of the timeline and test on the final 20%." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "split_idx = int(len(df) * 0.80)\n", "\n", "X_train = X.iloc[:split_idx].copy()\n", "X_test = X.iloc[split_idx:].copy()\n", "y_train = y.iloc[:split_idx].copy()\n", "y_test = y.iloc[split_idx:].copy()\n", "\n", "test_dates = df_raw.iloc[split_idx:]['dteday']\n", "\n", "print(f\"Train: X={X_train.shape}, y={y_train.shape}\")\n", "print(f\"Test : X={X_test.shape}, y={y_test.shape}\")\n", "print(f\"Test date range: {test_dates.min().date()} to {test_dates.max().date()}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 6. Build a Leakage-Safe Transform Pipeline\n", "\n", "The categorical fields are encoded with one-hot encoding. The continuous weather fields are robust-scaled. The pipeline is fitted on training data only, then replayed on the test data." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "categorical_cols = ['season', 'yr', 'mnth', 'hr', 'holiday', 'weekday', 'workingday', 'weathersit']\n", "continuous_cols = ['temp', 'atemp', 'hum', 'windspeed']\n", "\n", "pipeline = TransformPipeline(name='bike_sharing_regression_pipeline')\n", "pipeline.add('categorical', 'onehot_encode', columns=categorical_cols, sparse=False)\n", "pipeline.add('numerical', 'robust_scale', columns=continuous_cols)\n", "\n", "X_train_t = pipeline.fit_transform(X_train, y=y_train, verbose=True)\n", "X_test_t = pipeline.transform(X_test)\n", "\n", "pipeline_path = 'pipeline_regression_bike_sharing.pkl'\n", "pipeline.save(pipeline_path)\n", "\n", "print(f\"Transformed train shape: {X_train_t.shape}\")\n", "print(f\"Transformed test shape : {X_test_t.shape}\")\n", "print(f\"Saved pipeline : {pipeline_path}\")\n", "display(X_train_t.head())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 7. Train a Simple Baseline\n", "\n", "A serious regression workflow starts with a baseline. Ridge regression is not expected to dominate this problem, but it gives us a fast, stable reference point." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "ridge_config = TrainConfig(\n", " name='bike_demand_ridge_baseline',\n", " task='regression',\n", " model_type='ridge',\n", " optimizer='manual',\n", " model_params={'alpha': 1.0},\n", " optimization_metric='rmse',\n", " verbose=False,\n", " generate_reports=False,\n", " random_state=42,\n", ")\n", "\n", "ridge_trainer = OptunaTrainer(ridge_config)\n", "ridge_model = ridge_trainer.fit(X_train_t, y_train)\n", "ridge_pred = ridge_model.predict(X_test_t[ridge_trainer.state.selected_features])\n", "ridge_report = evaluate_regression(y_test, ridge_pred, n_features=X_test_t.shape[1], include_values=False)\n", "\n", "print('Ridge baseline metrics:')\n", "display(pd.Series(ridge_report.metrics).to_frame('value'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 8. Train an Optimized Regression Model\n", "\n", "Now we train a LightGBM regressor using the same `TrainConfig` structure used for classification. The regression-specific decisions are explicit:\n", "\n", "- `task='regression'`\n", "- `optimization_metric='rmse'`\n", "- `cv_strategy='time_series'`\n", "- threshold optimization is disabled automatically because regression predictions are continuous\n", "\n", "We also use regression-aware mutual information feature selection to keep the model focused on the strongest transformed predictors." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "lgbm_config = TrainConfig(\n", " name='bike_demand_lgbm',\n", " task='regression',\n", " model_type='lgbm',\n", " optimization_metric='rmse',\n", " optuna_sampler='tpe',\n", " n_trials=10,\n", " cv_folds=3,\n", " cv_strategy='time_series',\n", " feature_selection='mutual_info',\n", " feature_selection_params={'top_k': 35},\n", " min_features=10,\n", " use_early_stopping=False,\n", " save_feature_importance=True,\n", " generate_shap=False,\n", " verbose=True,\n", " optuna_show_progress=False,\n", " random_state=42,\n", ")\n", "\n", "t0 = time.time()\n", "lgbm_trainer = OptunaTrainer(lgbm_config)\n", "lgbm_model = lgbm_trainer.fit(X_train_t, y_train)\n", "lgbm_state = lgbm_trainer.state\n", "\n", "print(f\"\\nTraining finished in {time.time() - t0:.1f}s\")\n", "print(lgbm_state.summary())\n", "print(f\"Selected features: {lgbm_state.selected_features}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 9. Evaluate on the Hold-Out Test Set\n", "\n", "Cross-validation scores guide training. The hold-out test set tells us how the artifact behaves on later, unseen time periods. `evaluate_regression` returns JSON-friendly metrics and summaries that can be written directly into model metadata." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "model_features = lgbm_state.selected_features\n", "X_test_model = X_test_t[model_features]\n", "lgbm_pred = lgbm_model.predict(X_test_model)\n", "\n", "test_report = evaluate_regression(\n", " y_true=y_test,\n", " y_pred=lgbm_pred,\n", " n_features=len(model_features),\n", " include_values=True,\n", ")\n", "\n", "metrics_df = pd.Series(test_report.metrics).to_frame('value')\n", "print('LightGBM hold-out metrics:')\n", "display(metrics_df)\n", "\n", "comparison = pd.DataFrame({\n", " 'date': test_dates.reset_index(drop=True),\n", " 'actual_rentals': y_test.reset_index(drop=True),\n", " 'predicted_rentals': lgbm_pred,\n", "})\n", "comparison['residual'] = comparison['actual_rentals'] - comparison['predicted_rentals']\n", "display(comparison.head(10))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fig, axes = plt.subplots(1, 2, figsize=(13, 4))\n", "\n", "axes[0].scatter(comparison['actual_rentals'], comparison['predicted_rentals'], alpha=0.30, s=14, color='#2563EB')\n", "limit = [comparison[['actual_rentals', 'predicted_rentals']].min().min(), comparison[['actual_rentals', 'predicted_rentals']].max().max()]\n", "axes[0].plot(limit, limit, color='#111827', linewidth=1)\n", "axes[0].set_title('Actual vs predicted rentals')\n", "axes[0].set_xlabel('Actual rentals')\n", "axes[0].set_ylabel('Predicted rentals')\n", "\n", "sns.histplot(comparison['residual'], bins=50, kde=True, ax=axes[1], color='#F59E0B')\n", "axes[1].axvline(0, color='#111827', linewidth=1)\n", "axes[1].set_title('Residual distribution')\n", "axes[1].set_xlabel('Actual - predicted rentals')\n", "\n", "plt.tight_layout()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 10. Inspect Feature Importance\n", "\n", "For tree models, BitBullet stores model feature importance in the training state. Treat it as a diagnostic, not a final causal explanation: high importance means the model used that feature heavily, not that the feature causes demand to change." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "if lgbm_state.feature_importance is not None:\n", " display(lgbm_state.feature_importance.head(20))\n", " ax = lgbm_state.feature_importance.head(20).sort_values('importance').plot(\n", " x='feature', y='importance', kind='barh', figsize=(9, 6), legend=False, color='#10B981'\n", " )\n", " ax.set_title('Top LightGBM feature importances')\n", " ax.set_xlabel('Importance')\n", " ax.set_ylabel('Feature')\n", "else:\n", " print('No feature importance available for this model type.')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 11. Compare Model Families\n", "\n", "The same `TrainConfig` interface works across regression model families. This keeps experimentation focused on modeling decisions rather than library-specific boilerplate." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "benchmark_specs = {\n", " 'lgbm': {'n_trials': 5},\n", " 'xgb': {'n_trials': 5},\n", " 'random_forest': {'n_trials': 5},\n", " 'ridge': {'n_trials': 8},\n", "}\n", "\n", "leaderboard = []\n", "\n", "for model_type, spec in benchmark_specs.items():\n", " print(f\"\\nTraining {model_type}...\")\n", " config = TrainConfig(\n", " name=f'bike_demand_{model_type}',\n", " task='regression',\n", " model_type=model_type,\n", " optimization_metric='rmse',\n", " optuna_sampler='tpe',\n", " n_trials=spec['n_trials'],\n", " cv_folds=3,\n", " cv_strategy='time_series',\n", " use_early_stopping=False,\n", " save_feature_importance=False,\n", " generate_reports=False,\n", " verbose=False,\n", " optuna_show_progress=False,\n", " random_state=42,\n", " )\n", " trainer = OptunaTrainer(config)\n", " model = trainer.fit(X_train_t, y_train)\n", " features = trainer.state.selected_features\n", " pred = model.predict(X_test_t[features])\n", " report = evaluate_regression(y_test, pred, n_features=len(features), include_values=False)\n", "\n", " leaderboard.append({\n", " 'model_type': model_type,\n", " 'cv_rmse': trainer.state.best_score,\n", " 'test_rmse': report.metrics['rmse'],\n", " 'test_mae': report.metrics['mae'],\n", " 'test_r2': report.metrics['r2'],\n", " })\n", "\n", "leaderboard_df = pd.DataFrame(leaderboard).sort_values('test_rmse').reset_index(drop=True)\n", "display(leaderboard_df)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 12. Save the Model with Reproducibility Metadata\n", "\n", "A regression artifact needs more than model weights. It needs feature order, preprocessing dependency, target meaning, training config, evaluation numbers, and enough dataset metadata to audit what happened later.\n", "\n", "The fitted transform pipeline is saved separately, and the model metadata records that dependency. At inference time, raw rows must go through the same pipeline before prediction." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "models_dir = Path('models') / 'bike_sharing_demand_regressor' / '1.0.0'\n", "models_dir.mkdir(parents=True, exist_ok=True)\n", "model_path = models_dir / 'model.pkl'\n", "\n", "metadata = ModelMetadata(\n", " name='bike_sharing_demand_regressor',\n", " version='1.0.0',\n", " model_type=getattr(lgbm_model, 'model_type', type(lgbm_model).__name__),\n", " framework=getattr(lgbm_model, 'framework', 'sklearn'),\n", " task='regression',\n", " trained_at=datetime.now(),\n", " feature_names=model_features,\n", " n_features=len(model_features),\n", " hyperparameters=lgbm_state.best_params,\n", " training_time_seconds=lgbm_state.training_time_seconds,\n", " tags=['regression', 'bike_sharing', 'demand_forecasting', 'lightgbm', 'uci'],\n", " notes='Hourly bike-sharing demand regressor trained with BitBullet SDK.',\n", ")\n", "\n", "for metric_name, metric_value in test_report.metrics.items():\n", " if metric_value is not None:\n", " metadata.add_metric(f'test_{metric_name}', metric_value)\n", "\n", "metadata.add_feature_schema(X_train_t[model_features])\n", "metadata.add_cv_scores(lgbm_config.optimization_metric, lgbm_state.cv_scores or [lgbm_state.best_score])\n", "metadata.add_dataset(X_train_t[model_features], y_train, 'train')\n", "metadata.add_dataset(X_test_t[model_features], y_test, 'test')\n", "metadata.preprocessing = {\n", " 'pipeline_path': pipeline_path,\n", " 'pipeline_name': pipeline.name,\n", " 'fit_scope': 'train_only',\n", " 'raw_feature_names': X_train.columns.tolist(),\n", " 'transformed_feature_names': X_train_t.columns.tolist(),\n", "}\n", "metadata.search_metadata = {\n", " 'optimizer': lgbm_config.optimizer,\n", " 'optuna_sampler': lgbm_config.optuna_sampler,\n", " 'n_trials': lgbm_config.n_trials,\n", " 'cv_folds': lgbm_config.cv_folds,\n", " 'cv_strategy': lgbm_config.cv_strategy,\n", " 'optimization_metric': lgbm_config.optimization_metric,\n", " 'optimization_direction': lgbm_config.optimization_direction,\n", " 'best_score': lgbm_state.best_score,\n", " 'best_params': lgbm_state.best_params,\n", " 'selected_features': model_features,\n", "}\n", "metadata.inference_contract = {\n", " 'task': 'regression',\n", " 'target_name': TARGET,\n", " 'prediction_units': 'hourly_bike_rentals',\n", " 'preprocessing_required': True,\n", " 'pipeline_path': pipeline_path,\n", " 'feature_order': model_features,\n", " 'output': 'continuous_count_prediction',\n", " 'dropped_leakage_columns': leakage_cols,\n", "}\n", "metadata.add_artifact_metadata(regression_metrics=test_report.to_dict())\n", "\n", "ModelSerializer.save(\n", " model=lgbm_model,\n", " path=model_path,\n", " metadata=metadata,\n", " train_data=(X_train_t[model_features], y_train),\n", " test_data=(X_test_t[model_features], y_test),\n", " include_datasets=True,\n", ")\n", "\n", "print(f\"Model package saved: {model_path}\")\n", "print(f\"Metadata sidecar : {model_path.parent / (model_path.stem + '_metadata.json')}\")\n", "print()\n", "print(metadata.summary())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 13. Inference Replay\n", "\n", "Regression inference has the same rule as every supervised workflow: incoming raw rows must be transformed with the fitted pipeline, then ordered exactly as the model was trained. The model package tells us that feature order." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "loaded_pipeline = TransformPipeline.load(pipeline_path)\n", "loaded_package = ModelSerializer.load(model_path)\n", "\n", "new_rows_raw = X_test.iloc[:8].copy()\n", "new_rows_t = loaded_pipeline.transform(new_rows_raw)\n", "feature_order = loaded_package.metadata.inference_contract['feature_order']\n", "\n", "new_predictions = loaded_package.model.predict(new_rows_t[feature_order])\n", "\n", "inference_results = new_rows_raw.assign(\n", " predicted_rentals=np.maximum(0, new_predictions).round(0).astype(int),\n", " actual_rentals=y_test.iloc[:8].values,\n", ")\n", "\n", "display(inference_results[[\n", " 'season', 'hr', 'workingday', 'weathersit', 'temp', 'hum',\n", " 'predicted_rentals', 'actual_rentals'\n", "]])\n", "\n", "print('Inference replay complete: raw rows -> saved pipeline -> saved model -> ordered regression output.')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "\n", "## What You Built\n", "\n", "| You Wrote | BitBullet Handled |\n", "|-----------|-------------------|\n", "| `generate_feature_stats(X)` | Feature audit for mixed regression inputs |\n", "| Chronological train/test split | A more realistic demand-forecasting evaluation |\n", "| `pipeline.add(...)` | Repeatable transformation recipe |\n", "| `pipeline.fit_transform(X_train)` | Train-only parameter fitting |\n", "| `TrainConfig(task='regression')` | Regression-safe defaults and validation |\n", "| `OptunaTrainer(config).fit(...)` | TimeSeriesSplit CV, Optuna search, final model training |\n", "| `feature_selection='mutual_info'` | Regression-aware feature selection |\n", "| `evaluate_regression(...)` | RMSE, MAE, R2, adjusted R2, residual summaries, prediction summaries |\n", "| `ModelMetadata(...)` | Reproducibility, preprocessing, search, and inference metadata |\n", "| `ModelSerializer.save(...)` | Self-describing model package plus JSON sidecar |\n", "| `TransformPipeline.load(...)` and `ModelSerializer.load(...)` | Exact inference replay with feature order preserved |\n", "\n", "### Regression Checklist\n", "\n", "- Remove leakage columns before training.\n", "- Hold out test data before preprocessing.\n", "- Use a time-aware split for time-indexed demand data.\n", "- Use a simple baseline before optimization.\n", "- Choose metrics aligned to the business: RMSE penalizes large errors; MAE is easier to explain; R2 is relative fit quality.\n", "- Inspect residuals, not just headline metrics.\n", "- Save the transform pipeline and feature order with the model metadata.\n", "- Treat target units as part of the model contract.\n", "\n", "**Continue the Academy:**\n", "- `01_Binary_Classification.ipynb` - supervised classification workflow\n", "- `02_Multi_Class_Classification.ipynb` - multi-class classification workflow\n", "- `03_Clustering.ipynb` - unsupervised segmentation with `bitbullet.cluster`\n", "- `04_Data_Transformations.ipynb` - deep dive into transformation pipelines\n", "- `05_Model_Management.ipynb` - model metadata, serialization, and governance\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.13" } }, "nbformat": 4, "nbformat_minor": 5 }