Hier is een gedetailleerde implementatie voor hyperparameter tuning van je PyTorch-model: ## Aanbevolen Hyperparameters om te tunen: - **Leersnelheid**: 0.001 - 0.1 (logaritmische schaal) - **Momentum**: 0.8 - 0.99 - **Batchgrootte**: 32, 64, 128, 256 - **Aantal epochs**: (early stopping gebruiken) - **Weight decay**: 0.0001 - 0.01 - **Optimizer**: Adam, SGD, RMSprop ## Optuna Implementatie: ```python import optuna import torch import torch.nn as nn import torch.optim as optim from torch.utils.data import DataLoader def objective(trial): # Hyperparameter suggesties lr = trial.suggest_float('lr', 1e-3, 1e-1, log=True) momentum = trial.suggest_float('momentum', 0.8, 0.99) batch_size = trial.suggest_categorical('batch_size', [32, 64, 128]) weight_decay = trial.suggest_float('weight_decay', 1e-4, 1e-2, log=True) # Model en dataloaders model = YourCNNModel(num_classes=100) train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True) val_loader = DataLoader(val_dataset, batch_size=batch_size) # Optimizer en criterion optimizer = optim.SGD(model.parameters(), lr=lr, momentum=momentum, weight_decay=weight_decay) criterion = nn.CrossEntropyLoss() # Trainingsloop for epoch in range(50): # Max epochs met early stopping train_model(model, train_loader, optimizer, criterion) val_accuracy = evaluate_model(model, val_loader) # Rapporteer tussentijdse resultaten trial.report(val_accuracy, epoch) # Early stopping handling if trial.should_prune(): raise optuna.TrialPruned() return val_accuracy # Studie uitvoeren study = optuna.create_study(direction='maximize') study.optimize(objective, n_trials=100) # Beste parameters print("Beste parameters:", study.best_params) print("Beste accuracy:", study.best_value) ``` ## Ray Tune Implementatie: ```python from ray import tune from ray.tune import CLIReporter from ray.tune.schedulers import ASHAScheduler def train_model(config): model = YourCNNModel(num_classes=100) optimizer = optim.SGD(model.parameters(), lr=config['lr'], momentum=config['momentum'], weight_decay=config['weight_decay']) for epoch in range(50): # Trainingslogica train_epoch(model, train_loader, optimizer) val_accuracy = validate_model(model, val_loader) # Rapporteer naar Tune tune.report(accuracy=val_accuracy) # Configuratie space config = { 'lr': tune.loguniform(1e-3, 1e-1), 'momentum': tune.uniform(0.8, 0.99), 'batch_size': tune.choice([32, 64, 128]), 'weight_decay': tune.loguniform(1e-4, 1e-2) } # Scheduler voor early stopping scheduler = ASHAScheduler( metric='accuracy', mode='max', max_t=50, grace_period=10 ) reporter = CLIReporter(metric_columns=['accuracy']) # Run tuning analysis = tune.run( train_model, resources_per_trial={'cpu': 2, 'gpu': 1}, config=config, num_samples=100, scheduler=scheduler, progress_reporter=reporter ) ``` ## Integratie in Trainingsworkflow: 1. **Data Preparation**: ```python from sklearn.model_selection import train_test_split # Split dataset train_data, val_data = train_test_split(dataset, test_size=0.2, random_state=42) ``` 2. **Model Definitie**: ```python class YourCNNModel(nn.Module): def __init__(self, num_classes=100): super().__init__() self.features = nn.Sequential( nn.Conv2d(3, 32, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2), nn.Conv2d(32, 64, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2), nn.Conv2d(64, 128, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2) ) self.classifier = nn.Sequential( nn.Linear(128 * 4 * 4, 512), nn.ReLU(), nn.Dropout(0.5), nn.Linear(512, num_classes) ) ``` 3. **Efficiënte Tips**: - Gebruik **early stopping** om onnodige trials te voorkomen - Implementeer **model checkpointing** voor beste parameters - Gebruik **distributed tuning** voor parallelle executie - **Cache** voorbewerkte data voor snellere iteraties 4. **Resultaten Analyse**: ```python # Analyseer beste trial best_trial = study.best_trial print(f"Beste trial: {best_trial.number}") print(f"Beste value: {best_trial.value}") print(f"Beste params: {best_trial.params}") # Visualiseer resultaten optuna.visualization.plot_optimization_history(study) optuna.visualization.plot_param_importances(study) ``` ## Praktische Adviezen: 1. **Start met breed zoekbereik**, verfijn daarna 2. **Logaritmische schaal** voor leersnelheid 3. **Gebruik GPU-acceleratie** voor snellere training 4. **Monitor resources** om kosten te beheersen 5. **Valideer beste parameters** op testset Deze aanpak zal je helpen om optimale hyperparameters te vinden voor je CNN-model met 100 klassen en 50.000 afbeeldingen.