Skip to content

Main Pipeline

Main training pipeline with Hydra configuration management for food segmentation.

CLI and Hydra Configuration Management

This module demonstrates advanced CLI configuration management using Hydra and OmegaConf, providing flexible parameter override capabilities and structured configuration handling.

Key Configuration Features

Hydra Integration:

@hydra.main(version_base=None, config_path=config_dir, config_name="config")
def main(cfg: DictConfig):

Dynamic Configuration Loading: - Automatically resolves config directory from project root - Loads configuration from configs/config.yaml - Supports hierarchical configuration structures

CLI Usage Examples

Default Configuration:

python src/segmentation/main.py

Parameter Override:

python src/segmentation/main.py model.hyperparameters.epochs=50 model.hyperparameters.lr=0.001

Multiple Parameters:

python src/segmentation/main.py model.hyperparameters.epochs=100 model.hyperparameters.batch_size=32 paths.base_dir=/custom/path

Configuration Structure

The pipeline expects configuration with the following structure:

model:
  hyperparameters:
    epochs: 20
    batch_size: 16
    lr: 0.0001

paths:
  base_dir: "data/"

profiling:
  enabled: false

Enhanced User Experience

Rich Console Integration: - Displays formatted hyperparameter tables - Provides colorized progress indicators - Shows training status with visual panels

Automatic Environment Setup: - Configures Weights & Biases (wandb) in silent mode - Sets up proper Python path resolution - Handles project root discovery

Pipeline Workflow

  1. Configuration Loading - Hydra loads and validates config
  2. Parameter Display - Rich table shows current hyperparameters
  3. Data Loading - Initializes train/test data loaders
  4. Training Execution - Runs complete training pipeline
  5. Visualization Generation - Creates metrics and prediction plots
  6. Completion - Displays success confirmation

This approach provides reproducible experiments with easy parameter tuning through CLI overrides, making it ideal for hyperparameter sweeps and experiment tracking.

src.segmentation.main

main(cfg)

Main training pipeline for food segmentation model.

Orchestrates the complete training workflow including data loading, model training, and visualization generation. Uses Hydra for configuration management and Rich for enhanced console output.

Parameters:

Name Type Description Default
cfg DictConfig

Hydra configuration object containing all training parameters, paths, and settings. Expected structure: - cfg.model.hyperparameters: Training hyperparameters - cfg.paths.base_dir: Base directory for data and outputs - cfg.profiling.enabled: Whether to enable performance profiling

 required
Pipeline Steps
  1. Load and display configuration parameters
  2. Initialize data loaders for training and testing
  3. Create and configure trainer instance
  4. Execute training loop
  5. Generate training metrics visualizations
  6. Generate prediction visualizations
Example

Run with default config:

python main.py

Run with custom parameters:

python main.py model.hyperparameters.epochs=50 model.hyperparameters.lr=0.001

Source code in src/segmentation/main.py 
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
@hydra.main(version_base=None, config_path=config_dir, config_name="config")
def main(cfg: DictConfig):
    """
    Main training pipeline for food segmentation model.

    Orchestrates the complete training workflow including data loading,
    model training, and visualization generation. Uses Hydra for configuration
    management and Rich for enhanced console output.

    Args:
        cfg (DictConfig): Hydra configuration object containing all training
            parameters, paths, and settings. Expected structure:
            - cfg.model.hyperparameters: Training hyperparameters
            - cfg.paths.base_dir: Base directory for data and outputs
            - cfg.profiling.enabled: Whether to enable performance profiling

    Pipeline Steps:
        1. Load and display configuration parameters
        2. Initialize data loaders for training and testing
        3. Create and configure trainer instance
        4. Execute training loop
        5. Generate training metrics visualizations
        6. Generate prediction visualizations

    Example:
        Run with default config:
        >>> python main.py

        Run with custom parameters:
        >>> python main.py model.hyperparameters.epochs=50 model.hyperparameters.lr=0.001
    """
    epochs = cfg.model.hyperparameters.epochs
    batch_size = cfg.model.hyperparameters.batch_size
    learning_rate = cfg.model.hyperparameters.lr
    base_dir = cfg.paths.base_dir
    profiling_enabled = cfg.profiling.enabled
    print_hyperparameters_table(cfg)

    console.print(
        Panel.fit("🍕 Food Segmentation Training Pipeline", style="bold green")
    )
    console.print()

    console.print("[yellow]📊 Loading data...")
    train_loader, test_loader = data_loaders(
        base_dir=base_dir, batch_size=batch_size, num_workers=4
    )
    console.print(
        f"[green]✓ Train: {len(train_loader.dataset)} samples | Test: {len(test_loader.dataset)} samples[/green]"
    )
    console.print()

    console.print("[blue]🚀 Starting training...[/blue]")
    trainer = Trainer(
        epochs=epochs,
        base_dir=base_dir,
        batch_size=batch_size,
        lr=learning_rate,
        enable_profiler=profiling_enabled,
        init_wandb=True,
    )

    trainer.train()

    console.print("[magenta]📈 Generating visualizations...[/magenta]")
    trainer.visualize_training_metrics()
    trainer.visualize_predictions()

    console.print(
        Panel.fit("✅ Training Pipeline Completed Successfully!", style="bold green")
    )

print_hyperparameters_table(cfg)

Display hyperparameters in a clean table format using Rich.

Creates a formatted table showing key training configuration parameters including epochs, batch size, learning rate, and base directory path.

Parameters:

Name Type Description Default
cfg DictConfig

Hydra configuration object containing model hyperparameters and path settings. Expected to have structure: - cfg.model.hyperparameters.epochs - cfg.model.hyperparameters.batch_size - cfg.model.hyperparameters.lr - cfg.paths.base_dir

 required
Example

print_hyperparameters_table(cfg)

Displays a formatted table with training configuration

Source code in src/segmentation/main.py 
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
def print_hyperparameters_table(cfg: DictConfig):
    """
    Display hyperparameters in a clean table format using Rich.

    Creates a formatted table showing key training configuration parameters
    including epochs, batch size, learning rate, and base directory path.

    Args:
        cfg (DictConfig): Hydra configuration object containing model hyperparameters
            and path settings. Expected to have structure:
            - cfg.model.hyperparameters.epochs
            - cfg.model.hyperparameters.batch_size
            - cfg.model.hyperparameters.lr
            - cfg.paths.base_dir

    Example:
        >>> print_hyperparameters_table(cfg)
        # Displays a formatted table with training configuration
    """
    table = Table(
        title="🔧 Training Configuration", show_header=True, header_style="bold blue"
    )
    table.add_column("Parameter", style="cyan", width=20)
    table.add_column("Value", style="white", width=15)

    table.add_row("Epochs", str(cfg.model.hyperparameters.epochs))
    table.add_row("Batch Size", str(cfg.model.hyperparameters.batch_size))
    table.add_row("Learning Rate", str(cfg.model.hyperparameters.lr))
    table.add_row("Base Directory", str(cfg.paths.base_dir))

    console.print(table)
    console.print()