API Reference

class group_lasso.GroupLasso(groups=None, group_reg=0.05, l1_reg=0.05, n_iter=100, tol=1e-05, subsampling_scheme=None, fit_intercept=True, frobenius_lipschitz=False, random_state=None)

Sparse group lasso regularised least squares linear regression.

This class implements the Sparse Group Lasso [1] regularisation for linear regression with the mean squared penalty.

This class is implemented as both a regressor and a transformation. If the transform method is called, then the columns of the input that correspond to zero-valued regression coefficients are dropped.

The loss is optimised using the FISTA algorithm proposed in [2] with the generalised gradient-based restarting scheme proposed in [3]. This algorithm is not as accurate as a few other optimisation algorithms, but it is extremely efficient and does recover the sparsity patterns. We therefore reccomend that this class is used as a transformer to select the viable features and that the output is fed into another regression algorithm, such as RidgeRegression in scikit-learn.

References

[1] Simon, N., Friedman, J., Hastie, T., & Tibshirani, R. (2013). A sparse-group lasso. Journal of Computational and Graphical Statistics, 22(2), 231-245.

[2] Beck A, Teboulle M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences. 2009 Mar 4;2(1):183-202.

[3] O’Donoghue B, Candes E. (2015) Adaptive restart for accelerated gradient schemes. Foundations of computational mathematics. Jun 1;15(3):715-32

fit(X, y, lipschitz=None)

Fit a group lasso regularised linear regression model.

Parameters:

• X (np.ndarray) – Data matrix
• y (np.ndarray) – Target vector or matrix
• lipschitz (float or None [default=None]) – A Lipshitz bound for the mean squared loss with the given data and target matrices. If None, this is estimated.

fit_transform(X, y, lipschitz=None)

Fit a group lasso model to X and y and remove unused columns from X

get_params(deep=True)

Get parameters for this estimator.

Parameters:deep (boolean, optional) – If True, will return the parameters for this estimator and contained subobjects that are estimators. Returns:params – Parameter names mapped to their values. Return type:mapping of string to any

loss(X, y)

The group-lasso regularised loss with the current coefficients

Parameters:

• X (np.ndarray) – Data matrix, X.shape == (num_datapoints, num_features)
• y (np.ndarray) – Target vector/matrix, y.shape == (num_datapoints, num_targets), or y.shape == (num_datapoints,)

predict(X)

Predict using the linear model.

score(X, y, sample_weight=None)

Returns the coefficient of determination R^2 of the prediction.

The coefficient R^2 is defined as (1 - u/v), where u is the residual sum of squares ((y_true - y_pred) ** 2).sum() and v is the total sum of squares ((y_true - y_true.mean()) ** 2).sum(). The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a R^2 score of 0.0.

Parameters:

• X (array-like, shape = (n_samples, n_features)) – Test samples. For some estimators this may be a precomputed kernel matrix instead, shape = (n_samples, n_samples_fitted], where n_samples_fitted is the number of samples used in the fitting for the estimator.
• y (array-like, shape = (n_samples) or (n_samples, n_outputs)) – True values for X.
• sample_weight (array-like, shape = [n_samples], optional) – Sample weights.

Returns:

score – R^2 of self.predict(X) wrt. y.

Return type:

float

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Returns: Return type:self

sparsity_mask

A boolean mask indicating whether features are used in prediction.

subsample(*args)

Subsample the input using this class’s subsampling scheme.

transform(X)

Remove columns corresponding to zero-valued coefficients.

class group_lasso.LogisticGroupLasso(groups, group_reg=0.05, l1_reg=0.05, n_iter=100, tol=1e-05, subsampling_scheme=None, fit_intercept=True, random_state=None)

Sparse group lasso regularised single-class logistic regression.

This class implements the Sparse Group Lasso [1] regularisation for logistic regression with a cross entropy penalty.

This class is implemented as both a regressor and a transformation. If the transform method is called, then the columns of the input that correspond to zero-valued regression coefficients are dropped.

The loss is optimised using the FISTA algorithm proposed in [2] with the generalised gradient-based restarting scheme proposed in [3]. This algorithm is not as accurate as a few other optimisation algorithms, but it is extremely efficient and does recover the sparsity patterns. We therefore reccomend that this class is used as a transformer to select the viable features and that the output is fed into another classification algorithm, such as LogisticRegression in scikit-learn.

References

[1] Simon, N., Friedman, J., Hastie, T., & Tibshirani, R. (2013). A sparse-group lasso. Journal of Computational and Graphical Statistics, 22(2), 231-245.

[2] Beck A, Teboulle M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences. 2009 Mar 4;2(1):183-202.

[3] O’Donoghue B, Candes E. (2015) Adaptive restart for accelerated gradient schemes. Foundations of computational mathematics. Jun 1;15(3):715-32.

fit(X, y, lipschitz=None)

Fit a group-lasso regularised linear model.

fit_transform(X, y, lipschitz=None)

Fit a group lasso model to X and y and remove unused columns from X

get_params(deep=True)

Get parameters for this estimator.

Parameters:deep (boolean, optional) – If True, will return the parameters for this estimator and contained subobjects that are estimators. Returns:params – Parameter names mapped to their values. Return type:mapping of string to any

loss(X, y)

The group-lasso regularised loss with the current coefficients

Parameters:

• X (np.ndarray) – Data matrix, X.shape == (num_datapoints, num_features)
• y (np.ndarray) – Target vector/matrix, y.shape == (num_datapoints, num_targets), or y.shape == (num_datapoints,)

predict(X)

Predict using the linear model.

score(X, y, sample_weight=None)

Returns the mean accuracy on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters:

• X (array-like, shape = (n_samples, n_features)) – Test samples.
• y (array-like, shape = (n_samples) or (n_samples, n_outputs)) – True labels for X.
• sample_weight (array-like, shape = [n_samples], optional) – Sample weights.

Returns:

score – Mean accuracy of self.predict(X) wrt. y.

Return type:

float

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Returns: Return type:self

sparsity_mask

A boolean mask indicating whether features are used in prediction.

subsample(*args)

Subsample the input using this class’s subsampling scheme.

transform(X)

Remove columns corresponding to zero-valued coefficients.

class group_lasso.MultinomialGroupLasso(groups, group_reg=0.05, l1_reg=0.05, n_iter=100, tol=1e-05, subsampling_scheme=None, fit_intercept=True, random_state=None)

Sparse group lasso regularised multi-class logistic regression.

This class implements the Sparse Group Lasso [1] regularisation for multinomial regression (also known as multi-class logistic regression) with a cross entropy penalty.

This class is implemented as both a regressor and a transformation. If the transform method is called, then the columns of the input that correspond to zero-valued regression coefficients are dropped.

The loss is optimised using the FISTA algorithm proposed in [2] with the generalised gradient-based restarting scheme proposed in [3]. This algorithm is not as accurate as a few other optimisation algorithms, but it is extremely efficient and does recover the sparsity patterns. We therefore reccomend that this class is used as a transformer to select the viable features and that the output is fed into another classification algorithm, such as LogisticRegression in scikit-learn.

References

[1] Simon, N., Friedman, J., Hastie, T., & Tibshirani, R. (2013). A sparse-group lasso. Journal of Computational and Graphical Statistics, 22(2), 231-245.

[2] Beck A, Teboulle M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences. 2009 Mar 4;2(1):183-202.

[3] O’Donoghue B, Candes E. (2015) Adaptive restart for accelerated gradient schemes. Foundations of computational mathematics. Jun 1;15(3):715-32

fit(X, y, lipschitz=None)

Fit a group-lasso regularised linear model.

fit_transform(X, y, lipschitz=None)

Fit a group lasso model to X and y and remove unused columns from X

get_params(deep=True)

Get parameters for this estimator.

Parameters:deep (boolean, optional) – If True, will return the parameters for this estimator and contained subobjects that are estimators. Returns:params – Parameter names mapped to their values. Return type:mapping of string to any

loss(X, y)

The group-lasso regularised loss with the current coefficients

Parameters:

• X (np.ndarray) – Data matrix, X.shape == (num_datapoints, num_features)
• y (np.ndarray) – Target vector/matrix, y.shape == (num_datapoints, num_targets), or y.shape == (num_datapoints,)

predict(X)

Predict using the linear model.

score(X, y, sample_weight=None)

Returns the mean accuracy on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters:

• X (array-like, shape = (n_samples, n_features)) – Test samples.
• y (array-like, shape = (n_samples) or (n_samples, n_outputs)) – True labels for X.
• sample_weight (array-like, shape = [n_samples], optional) – Sample weights.

Returns:

score – Mean accuracy of self.predict(X) wrt. y.

Return type:

float

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Returns: Return type:self

sparsity_mask

A boolean mask indicating whether features are used in prediction.

subsample(*args)

Subsample the input using this class’s subsampling scheme.

transform(X)

Remove columns corresponding to zero-valued coefficients.