Feature Engineering and Selection for Regression Models with Python and Scikit-learn

Good regression models begin with careful feature preparation. This updated tutorial uses a local used-car listing dataset, current pandas and scikit-learn APIs, explicit data-quality checks, an interpretable vehicle-age feature, and preprocessing that is designed to be fitted on training rows only.

Load and validate the data

The notebook resolves data/car_prices/cars.csv relative to the project root, removes exact duplicates, and excludes rows without a positive target.

data = cars.drop_duplicates().copy()
data = data.loc[data["price_usd"].gt(0)].copy()

Missingness and unique-value counts reveal which columns need treatment and which high-cardinality fields may create an unnecessarily large encoded matrix.

Create vehicle age

SNAPSHOT_YEAR = 2019
data["vehicle_age"] = SNAPSHOT_YEAR - data["year_produced"]
data = data.loc[data["vehicle_age"].between(0, 80)].copy()

Vehicle age is easier to interpret than the raw production year. The listing snapshot year is fixed because this historical dataset was collected in 2019.

Define leakage-safe preprocessing

preprocessor = ColumnTransformer([
    ("numeric", Pipeline([
        ("imputer", SimpleImputer(strategy="median")),
        ("scaler", StandardScaler()),
    ]), numeric_features),
    ("categorical", Pipeline([
        ("imputer", SimpleImputer(strategy="most_frequent")),
        ("onehot", OneHotEncoder(handle_unknown="ignore", min_frequency=10)),
    ]), categorical_features),
])

The transformer should be placed inside the final model pipeline and fitted only after splitting the data. This prevents imputation, scaling, and category discovery from learning from the holdout.