TSNE

class cuml.manifold.TSNE(

*,

n_components=2,

perplexity=30.0,

early_exaggeration=12.0,

late_exaggeration=1.0,

learning_rate=200.0,

max_iter=1000,

n_iter_without_progress=300,

min_grad_norm=1e-07,

metric='euclidean',

metric_params=None,

init='random',

random_state=None,

method='fft',

angle=0.5,

n_neighbors=90,

perplexity_max_iter=100,

exaggeration_iter=250,

pre_momentum=0.5,

post_momentum=0.8,

learning_rate_method='adaptive',

square_distances=True,

precomputed_knn=None,

verbose=False,

output_type=None,

)

t-SNE (T-Distributed Stochastic Neighbor Embedding) is an extremely powerful dimensionality reduction technique that aims to maintain local distances between data points. It is extremely robust to whatever dataset you give it, and is used in many areas including cancer research, music analysis and neural network weight visualizations.

cuML’s t-SNE supports three algorithms: the original exact algorithm, the Barnes-Hut approximation and the fast Fourier transform interpolation approximation. The latter two are derived from CannyLabs’ open-source CUDA code and produce extremely fast embeddings when n_components = 2. The exact algorithm is more accurate, but too slow to use on large datasets.

Parameters:

n_componentsint (default 2)

The output dimensionality size. Currently only 2 is supported.

perplexityfloat (default 30.0)

Larger datasets require a larger value. Consider choosing different perplexity values from 5 to 50 and see the output differences.

early_exaggerationfloat (default 12.0)

Controls the space between clusters. Not critical to tune this.

late_exaggerationfloat (default 1.0)

Controls the space between clusters. It may be beneficial to increase this slightly to improve cluster separation. This will be applied after exaggeration_iter iterations (FFT only).

learning_ratefloat (default 200.0)

The learning rate usually between (10, 1000). If this is too high, t-SNE could look like a cloud / ball of points.

max_iterint (default 1000)

The more epochs, the more stable/accurate the final embedding.

n_iter_without_progressint (default 300)

Currently unused. When the KL Divergence becomes too small after some iterations, terminate t-SNE early.

min_grad_normfloat (default 1e-07)

The minimum gradient norm for when t-SNE will terminate early. Used in the ‘exact’ and ‘fft’ algorithms. Consider reducing if the embeddings are unsatisfactory. It’s recommended to use a smaller value for smaller datasets.

metricstr (default=’euclidean’).

Distance metric to use. Supported distances are [‘l1, ‘cityblock’, ‘manhattan’, ‘euclidean’, ‘l2’, ‘sqeuclidean’, ‘minkowski’, ‘chebyshev’, ‘cosine’, ‘correlation’]

initstr ‘random’ or ‘pca’ (default ‘random’)

Currently supports random or pca initialization.

verboseint or boolean, default=False

Sets logging level. It must be one of cuml.common.logger.level_*. See Verbosity Levels for more info.

random_stateint (default None)

Controls random initialization. For the FFT method, setting this value makes repeated runs with the same inputs deterministic.

methodstr ‘fft’, ‘barnes_hut’ or ‘exact’ (default ‘fft’)

‘barnes_hut’ and ‘fft’ are fast approximations. ‘exact’ is more accurate but slower.

anglefloat (default 0.5)

Valid values are between 0.0 and 1.0, which trade off speed and accuracy, respectively. Generally, these values are set between 0.2 and 0.8. (Barnes-Hut only.)

learning_rate_methodstr ‘adaptive’, ‘none’ or None (default ‘adaptive’)

Either adaptive or None. ‘adaptive’ tunes the learning rate, early exaggeration, perplexity and n_neighbors automatically based on input size.

n_neighborsint (default 90)

The number of datapoints you want to use in the attractive forces. Smaller values are better for preserving local structure, whilst larger values can improve global structure preservation. Default is 3 * 30 (perplexity)

perplexity_max_iterint (default 100)

The number of epochs the best gaussian bands are found for.

exaggeration_iterint (default 250)

To promote the growth of clusters, set this higher.

pre_momentumfloat (default 0.5)

During the exaggeration iteration, more forcefully apply gradients.

post_momentumfloat (default 0.8)

During the late phases, less forcefully apply gradients.

square_distancesboolean, default=True

Whether TSNE should square the distance values. Internally, this will be used to compute a kNN graph using the provided metric and then squaring it when True. If a knn_graph is passed to fit or fit_transform methods, all the distances will be squared when True. For example, if a knn_graph was obtained using ‘sqeuclidean’ metric, the distances will still be squared when True. Note: This argument should likely be set to False for distance metrics other than ‘euclidean’ and ‘l2’.

precomputed_knntuple[array, array], sparse-matrix, array, optional

This feature allows the precomputation of the KNN outside of TSNE. Options are:

• A tuple (indices, distances) of dense arrays of shape (n_samples, n_neighbors), where n_neighbors is >= the n_neighbors parameter. Self references should be included (i.e. the first column of indices should be [0, 1, …], denotating that the nearest neighbor to each row is itself). This is the most efficient representation.

• A sparse matrix KNN graph, as may be output by cuml.neighbors.kneighbors_graph with mode="distance" and include_self=True. The n_neighbors used to calculate the graph must be >= the n_neighbors parameter.

• A pairwise distances dense array of shape (n_samples, n_samples).

In all cases the KNN should be computed using the same metric as provided to TSNE.

output_type{None, ‘input’, ‘cupy’, ‘numpy’, ‘cudf’, ‘pandas’}, default=None

Return results and set estimator attributes to the indicated output type. If None, the output type set at the module level (cuml.global_settings.output_type) will be used. See Output Data Type Configuration for more info.

Attributes:

embedding_array

Stores the embedding vectors.

kl_divergence_float

TSNE.kl_divergence_(self)

learning_rate_float

Effective learning rate.

n_iter_int

Number of iterations run.

Methods

fit(self, X[, y, knn_graph])	Fit X into an embedded space.
fit_transform(self, X[, y, knn_graph])	Fit X into an embedded space and return that transformed output.

References

[1]

van der Maaten, L.J.P. t-Distributed Stochastic Neighbor Embedding

[2]

van der Maaten, L.J.P.; Hinton, G.E. Visualizing High-Dimensional Data Using t-SNE. Journal of Machine Learning Research 9:2579-2605, 2008.

[3]

George C. Linderman, Manas Rachh, Jeremy G. Hoskins, Stefan Steinerberger, Yuval Kluger Efficient Algorithms for t-distributed Stochastic Neighborhood Embedding

Tip

Maaten and Linderman showcased how t-SNE can be very sensitive to both the starting conditions (i.e. random initialization) and its hyperparameters. You can run t-SNE with different random seeds to compare embeddings and settle on the best configuration. For the FFT method, reusing the same random_state and inputs reproduces the same result.

Note

The CUDA implementation is derived from the excellent CannyLabs open source implementation here: CannyLab/tsne-cuda. The CannyLabs code is licensed according to the conditions in cuml/cpp/src/tsne/cannylabs_tsne_license.txt. A full description of their approach is available in their article t-SNE-CUDA: GPU-Accelerated t-SNE and its Applications to Modern Data (https://arxiv.org/abs/1807.11824).

fit(self, X, y=None, *, knn_graph=None) → 'TSNE'

Fit X into an embedded space.

Parameters:

Xarray-like (device or host) shape = (n_samples, n_features)

Dense or sparse matrix with dtype float32 or float64. Acceptable dense formats: CUDA array interface compliant objects like CuPy, cuDF DataFrame/Series, NumPy ndarray and Pandas DataFrame/Series.

yarray-like (device or host) shape = (n_samples, 1)

Dense matrix with dtype float32 or float64. Acceptable formats: CUDA array interface compliant objects like CuPy, cuDF DataFrame/Series, NumPy ndarray and Pandas DataFrame/Series.

htuple[array, array], sparse-matrix, array, optional

This feature allows the precomputation of the KNN outside of TSNE.

This may take any of the valid forms accepted by the precomputed_knn parameter to TSNE, and takes precedence over it. See the TSNE docstring on precomputed_knn for more information.

fit_transform(self, X, y=None, *, knn_graph=None)

Fit X into an embedded space and return that transformed output.

Parameters:

Xarray-like (device or host) shape = (n_samples, n_features)

Dense matrix with dtype float32 or float64. Acceptable formats: CUDA array interface compliant objects like CuPy, cuDF DataFrame/Series, NumPy ndarray and Pandas DataFrame/Series.

yarray-like (device or host) shape = (n_samples, 1)

Dense matrix with dtype float32 or float64. Acceptable formats: CUDA array interface compliant objects like CuPy, cuDF DataFrame/Series, NumPy ndarray and Pandas DataFrame/Series.

Returns:

X_newcuDF, CuPy or NumPy object depending on cuML’s output type configuration, shape = (n_samples, n_components)

Embedding of the data in low-dimensional space.

For more information on how to configure cuML’s output type, refer to: Output Data Type Configuration.

property kl_divergence_