# Hyperparameter Search with RandomizedSearchCV

This notebook demonstrates how `cuml.accel` speeds up a hyperparameter search workflow. Having your train of thought interrupted by long running steps in a workflow is not great. By using `cuml.accel` you can take a workflow that is tedious because it takes minutes to complete and make it complete in 30s.

In this example we build a preprocessing + classification pipeline and use `RandomizedSearchCV` to find the best configuration. However, the principle of using `cuml.accel` to take a task from “requires a coffee break per iteration” to “it is fun to iterate on ideas” by speeding it up applies to many other tasks as well.

**Pipeline:** `StandardScaler` → `PCA` → `KNeighborsClassifier`

KNN is distance-based, so the preprocessing steps are essential:

- `StandardScaler` normalises features that span very different ranges (elevation 0–3800 vs binary soil-type indicators 0/1).
- `PCA` reduces the 54-dimensional feature space (40 of which are sparse one-hot columns) to a compact representation where distances are more informative.

**Dataset:** Forest Cover Type (300K subsample, 54 features, 7 classes).

Without `cuml.accel`, this search takes several minutes (CPU, `n_jobs=10`). With `cuml.accel` enabled the same search completes in under a minute.

All three pipeline steps (`StandardScaler`, `PCA`, `KNeighborsClassifier`) are GPU-accelerated by `cuml.accel`.

```none
[1]:
```

## Load and prepare the dataset

We use the [Forest Cover Type](https://archive.ics.uci.edu/dataset/31/covertype) dataset (581K samples, 54 features, 7 cover-type classes). To keep runtimes manageable we subsample to 300K rows and split 80/20 into train and test sets.

```none
[2]:
```

## Define the pipeline and search space

The pipeline chains three steps, each GPU-accelerated by `cuml.accel`:

1. `StandardScaler` — normalise feature scales so that distance computations are not dominated by high-magnitude features like elevation.
2. `PCA` — project the 54 features (many of which are sparse one-hot indicators) into a lower-dimensional space.
3. `KNeighborsClassifier` — classify based on nearest neighbours in the PCA-reduced space.

We search over PCA dimensionality, number of neighbours, distance weighting, and distance metric.

```none
[3]:
```

## Run the search

We sample 20 random parameter combinations and evaluate each with 5-fold cross-validation, for a total of 100 pipeline fits. With `cuml.accel` active this takes ~30 seconds; without it (CPU, `n_jobs=10`) the same search takes ~4.5 minutes.

```none
[4]:
```

```none
[4]:
```

## Inspect the results

Let’s look at the best hyperparameters found by the search and how the top configurations compare.

```none
[5]:
```

```none
[6]:
```

```none
[6]:
```

## Evaluate on the test set

`RandomizedSearchCV` with `refit=True` automatically refits the best model on the full training set. We can use it directly to score on held-out data.

```none
[7]:
```