BitBullet:Lessons: Regression¶

Regression is the supervised learning task where the target is continuous: demand, price, revenue, risk exposure, delivery time, or any numeric outcome where the size of the error matters.

In this lesson we build a complete regression workflow with the BitBullet SDK: feature profiling, leakage-safe transformations, baseline training, hyperparameter search, residual analysis, model benchmarking, and model packaging with the metadata needed to replay inference correctly.

Dataset: Bike Sharing Dataset
Source: UCI Machine Learning Repository - Bike Sharing Dataset
File: hour.csv
Task: Predict the hourly number of rented bikes (cnt) from calendar, season, weather, and environmental features.

Before running this notebook, download the Bike Sharing Dataset and place hour.csv next to this notebook, or update data_path in Section 2.

---

What You Will Build¶

Step	Component	What BitBullet Handles For You
1	Feature profiling	generate_feature_stats for mixed numeric/categorical audit
2	Time-aware hold-out split	Train on earlier rows, test on later rows
3	Transform pipeline	Fit on train only, save, reload, and replay
4	Baseline model	Fixed-parameter Ridge regression through TrainConfig
5	Optimized model	LightGBM regression with Optuna and TimeSeriesSplit CV
6	Regression evaluation	evaluate_regression metrics, summaries, residuals, and predictions
7	Benchmarking	Compare model families using the same training interface
8	Model packaging	ModelMetadata and ModelSerializer for reproducible artifacts
9	Inference replay	Load pipeline + model, preserve feature order, predict new rows

1. Environment and Imports¶

In [ ]:

import sys
import os
import warnings
import time
from datetime import datetime
from pathlib import Path

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

warnings.filterwarnings('ignore')

# When running this notebook from bitbullet/lessons in a local clone, prefer the local SDK source.
sdk_root = os.path.abspath('..')
if sdk_root not in sys.path:
    sys.path.insert(0, sdk_root)

import bitbullet
print(f"bitbullet  : v{getattr(bitbullet, '__version__', 'dev')}")

from bitbullet.transform import TransformPipeline, generate_feature_stats
from bitbullet.train import TrainConfig, OptunaTrainer
from bitbullet.evaluate import evaluate_regression
from bitbullet.model import ModelMetadata, ModelSerializer

plt.rcParams.update({
    'figure.facecolor': 'white',
    'axes.spines.top': False,
    'axes.spines.right': False,
    'axes.grid': True,
    'grid.alpha': 0.25,
    'font.size': 11,
})

print('All imports successful.')

import sys import os import warnings import time from datetime import datetime from pathlib import Path import numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns warnings.filterwarnings('ignore') # When running this notebook from bitbullet/lessons in a local clone, prefer the local SDK source. sdk_root = os.path.abspath('..') if sdk_root not in sys.path: sys.path.insert(0, sdk_root) import bitbullet print(f"bitbullet : v{getattr(bitbullet, '__version__', 'dev')}") from bitbullet.transform import TransformPipeline, generate_feature_stats from bitbullet.train import TrainConfig, OptunaTrainer from bitbullet.evaluate import evaluate_regression from bitbullet.model import ModelMetadata, ModelSerializer plt.rcParams.update({ 'figure.facecolor': 'white', 'axes.spines.top': False, 'axes.spines.right': False, 'axes.grid': True, 'grid.alpha': 0.25, 'font.size': 11, }) print('All imports successful.')

2. Load the Dataset¶

Each row is one hour from the Capital Bikeshare system in Washington, D.C. The target cnt is the total number of rented bikes in that hour.

Two columns are leakage columns: casual and registered. They sum directly to cnt, so they must never be used as model inputs. We drop them before training.

In [ ]:

# Update this path if you stored the file in a different location.
data_path = 'hour.csv'

df_raw = pd.read_csv(data_path, parse_dates=['dteday'])
df_raw = df_raw.sort_values('instant').reset_index(drop=True)

print(f"Dataset loaded - shape: {df_raw.shape}")
print(f"Date range: {df_raw['dteday'].min().date()} to {df_raw['dteday'].max().date()}")
display(df_raw.head())

# Update this path if you stored the file in a different location. data_path = 'hour.csv' df_raw = pd.read_csv(data_path, parse_dates=['dteday']) df_raw = df_raw.sort_values('instant').reset_index(drop=True) print(f"Dataset loaded - shape: {df_raw.shape}") print(f"Date range: {df_raw['dteday'].min().date()} to {df_raw['dteday'].max().date()}") display(df_raw.head())

3. Prepare Features¶

The original dataset uses compact integer codes for season, weekday, hour, weather situation, and binary calendar flags. We convert these to categorical strings so the pipeline can encode them deliberately.

We keep dteday only for ordering and explanation. The model already receives month, hour, weekday, and year indicators.

In [ ]:

df = df_raw.copy()

season_map = {1: 'spring', 2: 'summer', 3: 'fall', 4: 'winter'}
weather_map = {
    1: 'clear_or_partly_cloudy',
    2: 'mist_or_cloudy',
    3: 'light_rain_or_snow',
    4: 'heavy_rain_or_snow',
}
weekday_map = {
    0: 'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday',
    4: 'thursday', 5: 'friday', 6: 'saturday',
}

df['season'] = df['season'].map(season_map)
df['weathersit'] = df['weathersit'].map(weather_map)
df['weekday'] = df['weekday'].map(weekday_map)
df['yr'] = df['yr'].map({0: '2011', 1: '2012'})

for col in ['mnth', 'hr', 'holiday', 'workingday']:
    df[col] = df[col].astype(str)

TARGET = 'cnt'
leakage_cols = ['casual', 'registered']
drop_cols = ['instant', 'dteday', TARGET] + leakage_cols

X = df.drop(columns=drop_cols)
y = df[TARGET]

print(f"Features: {X.shape[1]}")
print(f"Target range: {y.min()} to {y.max()} rentals per hour")
display(X.head())

df = df_raw.copy() season_map = {1: 'spring', 2: 'summer', 3: 'fall', 4: 'winter'} weather_map = { 1: 'clear_or_partly_cloudy', 2: 'mist_or_cloudy', 3: 'light_rain_or_snow', 4: 'heavy_rain_or_snow', } weekday_map = { 0: 'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday', 5: 'friday', 6: 'saturday', } df['season'] = df['season'].map(season_map) df['weathersit'] = df['weathersit'].map(weather_map) df['weekday'] = df['weekday'].map(weekday_map) df['yr'] = df['yr'].map({0: '2011', 1: '2012'}) for col in ['mnth', 'hr', 'holiday', 'workingday']: df[col] = df[col].astype(str) TARGET = 'cnt' leakage_cols = ['casual', 'registered'] drop_cols = ['instant', 'dteday', TARGET] + leakage_cols X = df.drop(columns=drop_cols) y = df[TARGET] print(f"Features: {X.shape[1]}") print(f"Target range: {y.min()} to {y.max()} rentals per hour") display(X.head())

4. Exploratory Feature Analysis¶

generate_feature_stats gives us a compact audit of missingness, skew, kurtosis, cardinality, and feature type. For regression, also inspect the target distribution because error metrics are shaped by the scale and skew of the target.

In [ ]:

stats_df, numerical_cols, categorical_cols = generate_feature_stats(X)

print(f"Numerical features   ({len(numerical_cols)}): {numerical_cols}")
print(f"Categorical features ({len(categorical_cols)}): {categorical_cols}")
print()
display(stats_df)

print('\nTarget summary:')
display(y.describe(percentiles=[0.05, 0.25, 0.5, 0.75, 0.95]))

stats_df, numerical_cols, categorical_cols = generate_feature_stats(X) print(f"Numerical features ({len(numerical_cols)}): {numerical_cols}") print(f"Categorical features ({len(categorical_cols)}): {categorical_cols}") print() display(stats_df) print('\nTarget summary:') display(y.describe(percentiles=[0.05, 0.25, 0.5, 0.75, 0.95]))

In [ ]:

fig, axes = plt.subplots(1, 2, figsize=(13, 4))

sns.histplot(y, bins=50, kde=True, ax=axes[0], color='#2563EB')
axes[0].set_title('Hourly rental count distribution')
axes[0].set_xlabel('Rentals per hour')

hourly_profile = df_raw.groupby('hr')['cnt'].mean()
hourly_profile.plot(ax=axes[1], color='#10B981', marker='o')
axes[1].set_title('Average rentals by hour')
axes[1].set_xlabel('Hour of day')
axes[1].set_ylabel('Average rentals')

plt.tight_layout()

fig, axes = plt.subplots(1, 2, figsize=(13, 4)) sns.histplot(y, bins=50, kde=True, ax=axes[0], color='#2563EB') axes[0].set_title('Hourly rental count distribution') axes[0].set_xlabel('Rentals per hour') hourly_profile = df_raw.groupby('hr')['cnt'].mean() hourly_profile.plot(ax=axes[1], color='#10B981', marker='o') axes[1].set_title('Average rentals by hour') axes[1].set_xlabel('Hour of day') axes[1].set_ylabel('Average rentals') plt.tight_layout()

5. Time-Aware Train/Test Split¶

This is demand over time, so a random split would make the test set unrealistically similar to the training set. We train on the first 80% of the timeline and test on the final 20%.

In [ ]:

split_idx = int(len(df) * 0.80)

X_train = X.iloc[:split_idx].copy()
X_test = X.iloc[split_idx:].copy()
y_train = y.iloc[:split_idx].copy()
y_test = y.iloc[split_idx:].copy()

test_dates = df_raw.iloc[split_idx:]['dteday']

print(f"Train: X={X_train.shape}, y={y_train.shape}")
print(f"Test : X={X_test.shape}, y={y_test.shape}")
print(f"Test date range: {test_dates.min().date()} to {test_dates.max().date()}")

split_idx = int(len(df) * 0.80) X_train = X.iloc[:split_idx].copy() X_test = X.iloc[split_idx:].copy() y_train = y.iloc[:split_idx].copy() y_test = y.iloc[split_idx:].copy() test_dates = df_raw.iloc[split_idx:]['dteday'] print(f"Train: X={X_train.shape}, y={y_train.shape}") print(f"Test : X={X_test.shape}, y={y_test.shape}") print(f"Test date range: {test_dates.min().date()} to {test_dates.max().date()}")

6. Build a Leakage-Safe Transform Pipeline¶

The categorical fields are encoded with one-hot encoding. The continuous weather fields are robust-scaled. The pipeline is fitted on training data only, then replayed on the test data.

In [ ]:

categorical_cols = ['season', 'yr', 'mnth', 'hr', 'holiday', 'weekday', 'workingday', 'weathersit']
continuous_cols = ['temp', 'atemp', 'hum', 'windspeed']

pipeline = TransformPipeline(name='bike_sharing_regression_pipeline')
pipeline.add('categorical', 'onehot_encode', columns=categorical_cols, sparse=False)
pipeline.add('numerical', 'robust_scale', columns=continuous_cols)

X_train_t = pipeline.fit_transform(X_train, y=y_train, verbose=True)
X_test_t = pipeline.transform(X_test)

pipeline_path = 'pipeline_regression_bike_sharing.pkl'
pipeline.save(pipeline_path)

print(f"Transformed train shape: {X_train_t.shape}")
print(f"Transformed test shape : {X_test_t.shape}")
print(f"Saved pipeline        : {pipeline_path}")
display(X_train_t.head())

categorical_cols = ['season', 'yr', 'mnth', 'hr', 'holiday', 'weekday', 'workingday', 'weathersit'] continuous_cols = ['temp', 'atemp', 'hum', 'windspeed'] pipeline = TransformPipeline(name='bike_sharing_regression_pipeline') pipeline.add('categorical', 'onehot_encode', columns=categorical_cols, sparse=False) pipeline.add('numerical', 'robust_scale', columns=continuous_cols) X_train_t = pipeline.fit_transform(X_train, y=y_train, verbose=True) X_test_t = pipeline.transform(X_test) pipeline_path = 'pipeline_regression_bike_sharing.pkl' pipeline.save(pipeline_path) print(f"Transformed train shape: {X_train_t.shape}") print(f"Transformed test shape : {X_test_t.shape}") print(f"Saved pipeline : {pipeline_path}") display(X_train_t.head())

7. Train a Simple Baseline¶

A serious regression workflow starts with a baseline. Ridge regression is not expected to dominate this problem, but it gives us a fast, stable reference point.

In [ ]:

ridge_config = TrainConfig(
    name='bike_demand_ridge_baseline',
    task='regression',
    model_type='ridge',
    optimizer='manual',
    model_params={'alpha': 1.0},
    optimization_metric='rmse',
    verbose=False,
    generate_reports=False,
    random_state=42,
)

ridge_trainer = OptunaTrainer(ridge_config)
ridge_model = ridge_trainer.fit(X_train_t, y_train)
ridge_pred = ridge_model.predict(X_test_t[ridge_trainer.state.selected_features])
ridge_report = evaluate_regression(y_test, ridge_pred, n_features=X_test_t.shape[1], include_values=False)

print('Ridge baseline metrics:')
display(pd.Series(ridge_report.metrics).to_frame('value'))

ridge_config = TrainConfig( name='bike_demand_ridge_baseline', task='regression', model_type='ridge', optimizer='manual', model_params={'alpha': 1.0}, optimization_metric='rmse', verbose=False, generate_reports=False, random_state=42, ) ridge_trainer = OptunaTrainer(ridge_config) ridge_model = ridge_trainer.fit(X_train_t, y_train) ridge_pred = ridge_model.predict(X_test_t[ridge_trainer.state.selected_features]) ridge_report = evaluate_regression(y_test, ridge_pred, n_features=X_test_t.shape[1], include_values=False) print('Ridge baseline metrics:') display(pd.Series(ridge_report.metrics).to_frame('value'))

8. Train an Optimized Regression Model¶

Now we train a LightGBM regressor using the same TrainConfig structure used for classification. The regression-specific decisions are explicit:

• task='regression'
• optimization_metric='rmse'
• cv_strategy='time_series'
• threshold optimization is disabled automatically because regression predictions are continuous

We also use regression-aware mutual information feature selection to keep the model focused on the strongest transformed predictors.

In [ ]:

lgbm_config = TrainConfig(
    name='bike_demand_lgbm',
    task='regression',
    model_type='lgbm',
    optimization_metric='rmse',
    optuna_sampler='tpe',
    n_trials=10,
    cv_folds=3,
    cv_strategy='time_series',
    feature_selection='mutual_info',
    feature_selection_params={'top_k': 35},
    min_features=10,
    use_early_stopping=False,
    save_feature_importance=True,
    generate_shap=False,
    verbose=True,
    optuna_show_progress=False,
    random_state=42,
)

t0 = time.time()
lgbm_trainer = OptunaTrainer(lgbm_config)
lgbm_model = lgbm_trainer.fit(X_train_t, y_train)
lgbm_state = lgbm_trainer.state

print(f"\nTraining finished in {time.time() - t0:.1f}s")
print(lgbm_state.summary())
print(f"Selected features: {lgbm_state.selected_features}")

lgbm_config = TrainConfig( name='bike_demand_lgbm', task='regression', model_type='lgbm', optimization_metric='rmse', optuna_sampler='tpe', n_trials=10, cv_folds=3, cv_strategy='time_series', feature_selection='mutual_info', feature_selection_params={'top_k': 35}, min_features=10, use_early_stopping=False, save_feature_importance=True, generate_shap=False, verbose=True, optuna_show_progress=False, random_state=42, ) t0 = time.time() lgbm_trainer = OptunaTrainer(lgbm_config) lgbm_model = lgbm_trainer.fit(X_train_t, y_train) lgbm_state = lgbm_trainer.state print(f"\nTraining finished in {time.time() - t0:.1f}s") print(lgbm_state.summary()) print(f"Selected features: {lgbm_state.selected_features}")

9. Evaluate on the Hold-Out Test Set¶

Cross-validation scores guide training. The hold-out test set tells us how the artifact behaves on later, unseen time periods. evaluate_regression returns JSON-friendly metrics and summaries that can be written directly into model metadata.

In [ ]:

model_features = lgbm_state.selected_features
X_test_model = X_test_t[model_features]
lgbm_pred = lgbm_model.predict(X_test_model)

test_report = evaluate_regression(
    y_true=y_test,
    y_pred=lgbm_pred,
    n_features=len(model_features),
    include_values=True,
)

metrics_df = pd.Series(test_report.metrics).to_frame('value')
print('LightGBM hold-out metrics:')
display(metrics_df)

comparison = pd.DataFrame({
    'date': test_dates.reset_index(drop=True),
    'actual_rentals': y_test.reset_index(drop=True),
    'predicted_rentals': lgbm_pred,
})
comparison['residual'] = comparison['actual_rentals'] - comparison['predicted_rentals']
display(comparison.head(10))

model_features = lgbm_state.selected_features X_test_model = X_test_t[model_features] lgbm_pred = lgbm_model.predict(X_test_model) test_report = evaluate_regression( y_true=y_test, y_pred=lgbm_pred, n_features=len(model_features), include_values=True, ) metrics_df = pd.Series(test_report.metrics).to_frame('value') print('LightGBM hold-out metrics:') display(metrics_df) comparison = pd.DataFrame({ 'date': test_dates.reset_index(drop=True), 'actual_rentals': y_test.reset_index(drop=True), 'predicted_rentals': lgbm_pred, }) comparison['residual'] = comparison['actual_rentals'] - comparison['predicted_rentals'] display(comparison.head(10))

In [ ]:

fig, axes = plt.subplots(1, 2, figsize=(13, 4))

axes[0].scatter(comparison['actual_rentals'], comparison['predicted_rentals'], alpha=0.30, s=14, color='#2563EB')
limit = [comparison[['actual_rentals', 'predicted_rentals']].min().min(), comparison[['actual_rentals', 'predicted_rentals']].max().max()]
axes[0].plot(limit, limit, color='#111827', linewidth=1)
axes[0].set_title('Actual vs predicted rentals')
axes[0].set_xlabel('Actual rentals')
axes[0].set_ylabel('Predicted rentals')

sns.histplot(comparison['residual'], bins=50, kde=True, ax=axes[1], color='#F59E0B')
axes[1].axvline(0, color='#111827', linewidth=1)
axes[1].set_title('Residual distribution')
axes[1].set_xlabel('Actual - predicted rentals')

plt.tight_layout()

fig, axes = plt.subplots(1, 2, figsize=(13, 4)) axes[0].scatter(comparison['actual_rentals'], comparison['predicted_rentals'], alpha=0.30, s=14, color='#2563EB') limit = [comparison[['actual_rentals', 'predicted_rentals']].min().min(), comparison[['actual_rentals', 'predicted_rentals']].max().max()] axes[0].plot(limit, limit, color='#111827', linewidth=1) axes[0].set_title('Actual vs predicted rentals') axes[0].set_xlabel('Actual rentals') axes[0].set_ylabel('Predicted rentals') sns.histplot(comparison['residual'], bins=50, kde=True, ax=axes[1], color='#F59E0B') axes[1].axvline(0, color='#111827', linewidth=1) axes[1].set_title('Residual distribution') axes[1].set_xlabel('Actual - predicted rentals') plt.tight_layout()

10. Inspect Feature Importance¶

For tree models, BitBullet stores model feature importance in the training state. Treat it as a diagnostic, not a final causal explanation: high importance means the model used that feature heavily, not that the feature causes demand to change.

In [ ]:

if lgbm_state.feature_importance is not None:
    display(lgbm_state.feature_importance.head(20))
    ax = lgbm_state.feature_importance.head(20).sort_values('importance').plot(
        x='feature', y='importance', kind='barh', figsize=(9, 6), legend=False, color='#10B981'
    )
    ax.set_title('Top LightGBM feature importances')
    ax.set_xlabel('Importance')
    ax.set_ylabel('Feature')
else:
    print('No feature importance available for this model type.')

if lgbm_state.feature_importance is not None: display(lgbm_state.feature_importance.head(20)) ax = lgbm_state.feature_importance.head(20).sort_values('importance').plot( x='feature', y='importance', kind='barh', figsize=(9, 6), legend=False, color='#10B981' ) ax.set_title('Top LightGBM feature importances') ax.set_xlabel('Importance') ax.set_ylabel('Feature') else: print('No feature importance available for this model type.')

11. Compare Model Families¶

The same TrainConfig interface works across regression model families. This keeps experimentation focused on modeling decisions rather than library-specific boilerplate.

In [ ]:

benchmark_specs = {
    'lgbm': {'n_trials': 5},
    'xgb': {'n_trials': 5},
    'random_forest': {'n_trials': 5},
    'ridge': {'n_trials': 8},
}

leaderboard = []

for model_type, spec in benchmark_specs.items():
    print(f"\nTraining {model_type}...")
    config = TrainConfig(
        name=f'bike_demand_{model_type}',
        task='regression',
        model_type=model_type,
        optimization_metric='rmse',
        optuna_sampler='tpe',
        n_trials=spec['n_trials'],
        cv_folds=3,
        cv_strategy='time_series',
        use_early_stopping=False,
        save_feature_importance=False,
        generate_reports=False,
        verbose=False,
        optuna_show_progress=False,
        random_state=42,
    )
    trainer = OptunaTrainer(config)
    model = trainer.fit(X_train_t, y_train)
    features = trainer.state.selected_features
    pred = model.predict(X_test_t[features])
    report = evaluate_regression(y_test, pred, n_features=len(features), include_values=False)

    leaderboard.append({
        'model_type': model_type,
        'cv_rmse': trainer.state.best_score,
        'test_rmse': report.metrics['rmse'],
        'test_mae': report.metrics['mae'],
        'test_r2': report.metrics['r2'],
    })

leaderboard_df = pd.DataFrame(leaderboard).sort_values('test_rmse').reset_index(drop=True)
display(leaderboard_df)

benchmark_specs = { 'lgbm': {'n_trials': 5}, 'xgb': {'n_trials': 5}, 'random_forest': {'n_trials': 5}, 'ridge': {'n_trials': 8}, } leaderboard = [] for model_type, spec in benchmark_specs.items(): print(f"\nTraining {model_type}...") config = TrainConfig( name=f'bike_demand_{model_type}', task='regression', model_type=model_type, optimization_metric='rmse', optuna_sampler='tpe', n_trials=spec['n_trials'], cv_folds=3, cv_strategy='time_series', use_early_stopping=False, save_feature_importance=False, generate_reports=False, verbose=False, optuna_show_progress=False, random_state=42, ) trainer = OptunaTrainer(config) model = trainer.fit(X_train_t, y_train) features = trainer.state.selected_features pred = model.predict(X_test_t[features]) report = evaluate_regression(y_test, pred, n_features=len(features), include_values=False) leaderboard.append({ 'model_type': model_type, 'cv_rmse': trainer.state.best_score, 'test_rmse': report.metrics['rmse'], 'test_mae': report.metrics['mae'], 'test_r2': report.metrics['r2'], }) leaderboard_df = pd.DataFrame(leaderboard).sort_values('test_rmse').reset_index(drop=True) display(leaderboard_df)

12. Save the Model with Reproducibility Metadata¶

A regression artifact needs more than model weights. It needs feature order, preprocessing dependency, target meaning, training config, evaluation numbers, and enough dataset metadata to audit what happened later.

The fitted transform pipeline is saved separately, and the model metadata records that dependency. At inference time, raw rows must go through the same pipeline before prediction.

In [ ]:

models_dir = Path('models') / 'bike_sharing_demand_regressor' / '1.0.0'
models_dir.mkdir(parents=True, exist_ok=True)
model_path = models_dir / 'model.pkl'

metadata = ModelMetadata(
    name='bike_sharing_demand_regressor',
    version='1.0.0',
    model_type=getattr(lgbm_model, 'model_type', type(lgbm_model).__name__),
    framework=getattr(lgbm_model, 'framework', 'sklearn'),
    task='regression',
    trained_at=datetime.now(),
    feature_names=model_features,
    n_features=len(model_features),
    hyperparameters=lgbm_state.best_params,
    training_time_seconds=lgbm_state.training_time_seconds,
    tags=['regression', 'bike_sharing', 'demand_forecasting', 'lightgbm', 'uci'],
    notes='Hourly bike-sharing demand regressor trained with BitBullet SDK.',
)

for metric_name, metric_value in test_report.metrics.items():
    if metric_value is not None:
        metadata.add_metric(f'test_{metric_name}', metric_value)

metadata.add_feature_schema(X_train_t[model_features])
metadata.add_cv_scores(lgbm_config.optimization_metric, lgbm_state.cv_scores or [lgbm_state.best_score])
metadata.add_dataset(X_train_t[model_features], y_train, 'train')
metadata.add_dataset(X_test_t[model_features], y_test, 'test')
metadata.preprocessing = {
    'pipeline_path': pipeline_path,
    'pipeline_name': pipeline.name,
    'fit_scope': 'train_only',
    'raw_feature_names': X_train.columns.tolist(),
    'transformed_feature_names': X_train_t.columns.tolist(),
}
metadata.search_metadata = {
    'optimizer': lgbm_config.optimizer,
    'optuna_sampler': lgbm_config.optuna_sampler,
    'n_trials': lgbm_config.n_trials,
    'cv_folds': lgbm_config.cv_folds,
    'cv_strategy': lgbm_config.cv_strategy,
    'optimization_metric': lgbm_config.optimization_metric,
    'optimization_direction': lgbm_config.optimization_direction,
    'best_score': lgbm_state.best_score,
    'best_params': lgbm_state.best_params,
    'selected_features': model_features,
}
metadata.inference_contract = {
    'task': 'regression',
    'target_name': TARGET,
    'prediction_units': 'hourly_bike_rentals',
    'preprocessing_required': True,
    'pipeline_path': pipeline_path,
    'feature_order': model_features,
    'output': 'continuous_count_prediction',
    'dropped_leakage_columns': leakage_cols,
}
metadata.add_artifact_metadata(regression_metrics=test_report.to_dict())

ModelSerializer.save(
    model=lgbm_model,
    path=model_path,
    metadata=metadata,
    train_data=(X_train_t[model_features], y_train),
    test_data=(X_test_t[model_features], y_test),
    include_datasets=True,
)

print(f"Model package saved: {model_path}")
print(f"Metadata sidecar  : {model_path.parent / (model_path.stem + '_metadata.json')}")
print()
print(metadata.summary())

models_dir = Path('models') / 'bike_sharing_demand_regressor' / '1.0.0' models_dir.mkdir(parents=True, exist_ok=True) model_path = models_dir / 'model.pkl' metadata = ModelMetadata( name='bike_sharing_demand_regressor', version='1.0.0', model_type=getattr(lgbm_model, 'model_type', type(lgbm_model).__name__), framework=getattr(lgbm_model, 'framework', 'sklearn'), task='regression', trained_at=datetime.now(), feature_names=model_features, n_features=len(model_features), hyperparameters=lgbm_state.best_params, training_time_seconds=lgbm_state.training_time_seconds, tags=['regression', 'bike_sharing', 'demand_forecasting', 'lightgbm', 'uci'], notes='Hourly bike-sharing demand regressor trained with BitBullet SDK.', ) for metric_name, metric_value in test_report.metrics.items(): if metric_value is not None: metadata.add_metric(f'test_{metric_name}', metric_value) metadata.add_feature_schema(X_train_t[model_features]) metadata.add_cv_scores(lgbm_config.optimization_metric, lgbm_state.cv_scores or [lgbm_state.best_score]) metadata.add_dataset(X_train_t[model_features], y_train, 'train') metadata.add_dataset(X_test_t[model_features], y_test, 'test') metadata.preprocessing = { 'pipeline_path': pipeline_path, 'pipeline_name': pipeline.name, 'fit_scope': 'train_only', 'raw_feature_names': X_train.columns.tolist(), 'transformed_feature_names': X_train_t.columns.tolist(), } metadata.search_metadata = { 'optimizer': lgbm_config.optimizer, 'optuna_sampler': lgbm_config.optuna_sampler, 'n_trials': lgbm_config.n_trials, 'cv_folds': lgbm_config.cv_folds, 'cv_strategy': lgbm_config.cv_strategy, 'optimization_metric': lgbm_config.optimization_metric, 'optimization_direction': lgbm_config.optimization_direction, 'best_score': lgbm_state.best_score, 'best_params': lgbm_state.best_params, 'selected_features': model_features, } metadata.inference_contract = { 'task': 'regression', 'target_name': TARGET, 'prediction_units': 'hourly_bike_rentals', 'preprocessing_required': True, 'pipeline_path': pipeline_path, 'feature_order': model_features, 'output': 'continuous_count_prediction', 'dropped_leakage_columns': leakage_cols, } metadata.add_artifact_metadata(regression_metrics=test_report.to_dict()) ModelSerializer.save( model=lgbm_model, path=model_path, metadata=metadata, train_data=(X_train_t[model_features], y_train), test_data=(X_test_t[model_features], y_test), include_datasets=True, ) print(f"Model package saved: {model_path}") print(f"Metadata sidecar : {model_path.parent / (model_path.stem + '_metadata.json')}") print() print(metadata.summary())

13. Inference Replay¶

Regression inference has the same rule as every supervised workflow: incoming raw rows must be transformed with the fitted pipeline, then ordered exactly as the model was trained. The model package tells us that feature order.

In [ ]:

loaded_pipeline = TransformPipeline.load(pipeline_path)
loaded_package = ModelSerializer.load(model_path)

new_rows_raw = X_test.iloc[:8].copy()
new_rows_t = loaded_pipeline.transform(new_rows_raw)
feature_order = loaded_package.metadata.inference_contract['feature_order']

new_predictions = loaded_package.model.predict(new_rows_t[feature_order])

inference_results = new_rows_raw.assign(
    predicted_rentals=np.maximum(0, new_predictions).round(0).astype(int),
    actual_rentals=y_test.iloc[:8].values,
)

display(inference_results[[
    'season', 'hr', 'workingday', 'weathersit', 'temp', 'hum',
    'predicted_rentals', 'actual_rentals'
]])

print('Inference replay complete: raw rows -> saved pipeline -> saved model -> ordered regression output.')

loaded_pipeline = TransformPipeline.load(pipeline_path) loaded_package = ModelSerializer.load(model_path) new_rows_raw = X_test.iloc[:8].copy() new_rows_t = loaded_pipeline.transform(new_rows_raw) feature_order = loaded_package.metadata.inference_contract['feature_order'] new_predictions = loaded_package.model.predict(new_rows_t[feature_order]) inference_results = new_rows_raw.assign( predicted_rentals=np.maximum(0, new_predictions).round(0).astype(int), actual_rentals=y_test.iloc[:8].values, ) display(inference_results[[ 'season', 'hr', 'workingday', 'weathersit', 'temp', 'hum', 'predicted_rentals', 'actual_rentals' ]]) print('Inference replay complete: raw rows -> saved pipeline -> saved model -> ordered regression output.')

---

What You Built¶

You Wrote	BitBullet Handled
generate_feature_stats(X)	Feature audit for mixed regression inputs
Chronological train/test split	A more realistic demand-forecasting evaluation
pipeline.add(...)	Repeatable transformation recipe
pipeline.fit_transform(X_train)	Train-only parameter fitting
TrainConfig(task='regression')	Regression-safe defaults and validation
OptunaTrainer(config).fit(...)	TimeSeriesSplit CV, Optuna search, final model training
feature_selection='mutual_info'	Regression-aware feature selection
evaluate_regression(...)	RMSE, MAE, R2, adjusted R2, residual summaries, prediction summaries
ModelMetadata(...)	Reproducibility, preprocessing, search, and inference metadata
ModelSerializer.save(...)	Self-describing model package plus JSON sidecar
TransformPipeline.load(...) and ModelSerializer.load(...)	Exact inference replay with feature order preserved

Regression Checklist¶

• Remove leakage columns before training.
• Hold out test data before preprocessing.
• Use a time-aware split for time-indexed demand data.
• Use a simple baseline before optimization.
• Choose metrics aligned to the business: RMSE penalizes large errors; MAE is easier to explain; R2 is relative fit quality.
• Inspect residuals, not just headline metrics.
• Save the transform pipeline and feature order with the model metadata.
• Treat target units as part of the model contract.

Continue the Academy:

• 01_Binary_Classification.ipynb - supervised classification workflow
• 02_Multi_Class_Classification.ipynb - multi-class classification workflow
• 03_Clustering.ipynb - unsupervised segmentation with bitbullet.cluster
• 04_Data_Transformations.ipynb - deep dive into transformation pipelines
• 05_Model_Management.ipynb - model metadata, serialization, and governance