# SGD

### *class* cuml.solvers.SGD(, loss='squared_loss', penalty=None, alpha=0.0001, l1_ratio=0.15, fit_intercept=True, epochs=1000, tol=0.001, shuffle=True, learning_rate='constant', eta0=0.001, power_t=0.5, batch_size=32, n_iter_no_change=5, output_type=None, verbose=False)

Stochastic Gradient Descent is a very common machine learning algorithm
where one optimizes some cost function via gradient steps. This makes SGD
very attractive for large problems when the exact solution is hard or even
impossible to find.

cuML’s SGD algorithm accepts a numpy matrix or a cuDF DataFrame as the
input dataset. The SGD algorithm currently works with linear regression,
ridge regression and SVM models.

* **Parameters:**
  **loss**
  : ‘hinge’ uses linear SVM
    ‘log’ uses logistic regression
    ‘squared_loss’ uses linear regression

  **penalty**
  : The penalty (aka regularization term) to apply.
    - ‘l1’: L1 norm (Lasso) regularization
    - ‘l2’: L2 norm (Ridge) regularization
    - ‘elasticnet’: Elastic Net regularization, a weighted average of L1 and L2
    - None: no penalty is added (the default)

  **alpha**
  : The constant value which decides the degree of regularization

  **fit_intercept**
  : If True, the model tries to correct for the global mean of y.
    If False, the model expects that you have centered the data.

  **epochs**
  : The number of times the model should iterate through the entire dataset
    during training (default = 1000)

  **tol**
  : The training process will stop if current_loss > previous_loss - tol

  **shuffle**
  : True, shuffles the training data after each epoch
    False, does not shuffle the training data after each epoch

  **eta0**
  : Initial learning rate

  **power_t**
  : The exponent used for calculating the invscaling learning rate

  **batch_size**
  : The number of samples to use for each batch.

  **learning_rate**
  : constant keeps the learning rate constant
    adaptive changes the learning rate if the training loss or the
    validation accuracy does not improve for n_iter_no_change epochs.
    The old learning rate is generally divide by 5

  **n_iter_no_change**
  : The number of epochs to train without any improvement in the model

  **output_type**
  : Return results and set estimator attributes to the indicated output
    type. If None, the output type set at the module level
    (`cuml.global_settings.output_type`) will be used. See
    [Output Data Type Configuration](../../cuml/index.html.md#output-data-type-configuration) for more info.

  **verbose**
  : Sets logging level. It must be one of `cuml.common.logger.level_*`.
    See [Verbosity Levels](../../cuml/index.html.md#verbosity-levels) for more info.
* **Attributes:**
  **coef_**

### Methods

| [`fit`](#cuml.solvers.SGD.fit)(self, X, y)   | Fit the model with X and y.   |
|--------------------------------------------------------------------|-------------------------------|
| [`predict`](#cuml.solvers.SGD.predict)(self, X)  | Predicts the y for X.         |

### Examples

```python
>>> import numpy as np
>>> import cudf
>>> from cuml.solvers import SGD as cumlSGD
>>> X = cudf.DataFrame()
>>> X['col1'] = np.array([1,1,2,2], dtype=np.float32)
>>> X['col2'] = np.array([1,2,2,3], dtype=np.float32)
>>> y = cudf.Series(np.array([1, 1, 2, 2], dtype=np.float32))
>>> pred_data = cudf.DataFrame()
>>> pred_data['col1'] = np.asarray([3, 2], dtype=np.float32)
>>> pred_data['col2'] = np.asarray([5, 5], dtype=np.float32)
>>> cu_sgd = cumlSGD(learning_rate='constant', eta0=0.005, epochs=2000,
...                  fit_intercept=True, batch_size=2,
...                  tol=0.0, penalty=None, loss='squared_loss')
>>> cu_sgd.fit(X, y)
SGD(batch_size=2, epochs=2000, eta0=0.005, tol=0.0)
>>> cu_pred = cu_sgd.predict(pred_data).to_numpy()
>>> print(" cuML intercept : ", cu_sgd.intercept_)
cuML intercept :  0.00418...
>>> print(" cuML coef : ", cu_sgd.coef_)
cuML coef :  0      0.9841...
1      0.0097...
dtype: float32
>>> print("cuML predictions : ", cu_pred)
cuML predictions :  [3.0055...  2.0214...]
```

<!-- !! processed by numpydoc !! -->

#### fit(self, X, y) → 'SGD'[[source]](https://github.com/NVIDIA/cuml/blob/93418baf5/python/cuml/cuml/solvers/sgd.pyx#L406)

Fit the model with X and y.

* **Parameters:**
  **X**
  : Dense matrix with dtype float32 or float64.
    Acceptable formats: CUDA array interface compliant objects like
    CuPy, cuDF DataFrame/Series, NumPy ndarray and Pandas
    DataFrame/Series.

  **y**
  : Dense matrix with dtype float32 or float64.
    Acceptable formats: CUDA array interface compliant objects like
    CuPy, cuDF DataFrame/Series, NumPy ndarray and Pandas
    DataFrame/Series.

<!-- !! processed by numpydoc !! -->

#### get_params(deep=True)[[source]](https://github.com/NVIDIA/cuml/blob/93418baf5/python/cuml/cuml/internals/base.py#L191)

Returns a dict of all params owned by this class. If the child class
has appropriately overridden the `_get_param_names` method and does not
need anything other than what is there in this method, then it doesn’t
have to override this method

<!-- !! processed by numpydoc !! -->

#### predict(self, X)[[source]](https://github.com/NVIDIA/cuml/blob/93418baf5/python/cuml/cuml/solvers/sgd.pyx#L435)

Predicts the y for X.

* **Parameters:**
  **X**
  : Dense matrix with dtype float32 or float64.
    Acceptable formats: CUDA array interface compliant objects like
    CuPy, cuDF DataFrame/Series, NumPy ndarray and Pandas
    DataFrame/Series.
* **Returns:**
  **preds**
  : Predicted values
    <br/>
    For more information on how to configure cuML’s output type,
    refer to: [Output Data Type Configuration](../../cuml/index.html.md#output-data-type-configuration).

<!-- !! processed by numpydoc !! -->

#### set_params(\*\*params)[[source]](https://github.com/NVIDIA/cuml/blob/93418baf5/python/cuml/cuml/internals/base.py#L200)

Accepts a dict of params and updates the corresponding ones owned by
this class. If the child class has appropriately overridden the
`_get_param_names` method and does not need anything other than what is,
there in this method, then it doesn’t have to override this method

<!-- !! processed by numpydoc !! -->