Tuning Model Hyperparameters with Grid Search at the Example of Training a Random Forest Classifier in Python

Grid search evaluates every parameter combination in a predefined grid. This updated tutorial uses current scikit-learn APIs, stratified cross-validation, a probability-based scoring metric, and an untouched holdout.

Prepare the classification data

The example uses scikit-learn’s built-in breast-cancer dataset and preserves the class ratio in the train/test split.

X_train, X_test, y_train, y_test = train_test_split(
    dataset.data,
    dataset.target,
    test_size=0.25,
    stratify=dataset.target,
    random_state=RANDOM_SEED,
)

Configure grid search

search = GridSearchCV(
    RandomForestClassifier(
        class_weight="balanced", random_state=RANDOM_SEED, n_jobs=-1
    ),
    param_grid=parameter_grid,
    scoring="average_precision",
    cv=StratifiedKFold(n_splits=4, shuffle=True, random_state=RANDOM_SEED),
    n_jobs=1,
)
search.fit(X_train, y_train)

Average precision is preferable to accuracy when probability ranking and minority-class detection matter. The selected model is also compared with a DummyClassifier prevalence baseline.

Evaluate the result

The final average precision, ROC AUC, classification report, and confusion matrix are calculated once on the untouched holdout.

A small exhaustive grid is appropriate for teaching. Larger production searches should reflect a justified compute budget, use nested or repeated validation where needed, and include threshold selection based on real error costs.