API Reference¶
-
class
group_lasso.
GroupLasso
(groups=None, group_reg=0.05, l1_reg=0.05, n_iter=100, tol=1e-05, subsampling_scheme=None, fit_intercept=True, frobenius_lipschitz=False, random_state=None)[source]¶ Sparse group lasso regularised least squares linear regression.
This class implements the Sparse Group Lasso [1] regularisation for linear regression with the mean squared penalty.
This class is implemented as both a regressor and a transformation. If the
transform
method is called, then the columns of the input that correspond to zero-valued regression coefficients are dropped.The loss is optimised using the FISTA algorithm proposed in [2] with the generalised gradient-based restarting scheme proposed in [3]. This algorithm is not as accurate as a few other optimisation algorithms, but it is extremely efficient and does recover the sparsity patterns. We therefore reccomend that this class is used as a transformer to select the viable features and that the output is fed into another regression algorithm, such as RidgeRegression in scikit-learn.
References
[1] Simon, N., Friedman, J., Hastie, T., & Tibshirani, R. (2013). A sparse-group lasso. Journal of Computational and Graphical Statistics, 22(2), 231-245.
[2] Beck A, Teboulle M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences. 2009 Mar 4;2(1):183-202.
[3] O’Donoghue B, Candes E. (2015) Adaptive restart for accelerated gradient schemes. Foundations of computational mathematics. Jun 1;15(3):715-32
-
fit
(X, y, lipschitz=None)[source]¶ Fit a group lasso regularised linear regression model.
Parameters: - X (np.ndarray) – Data matrix
- y (np.ndarray) – Target vector or matrix
- lipschitz (float or None [default=None]) – A Lipshitz bound for the mean squared loss with the given data and target matrices. If None, this is estimated.
-
fit_transform
(X, y, lipschitz=None)¶ Fit a group lasso model to X and y and remove unused columns from X
-
get_params
(deep=True)¶ Get parameters for this estimator.
Parameters: deep (boolean, optional) – If True, will return the parameters for this estimator and contained subobjects that are estimators. Returns: params – Parameter names mapped to their values. Return type: mapping of string to any
-
loss
(X, y)¶ The group-lasso regularised loss with the current coefficients
Parameters: - X (np.ndarray) – Data matrix,
X.shape == (num_datapoints, num_features)
- y (np.ndarray) – Target vector/matrix,
y.shape == (num_datapoints, num_targets)
, ory.shape == (num_datapoints,)
- X (np.ndarray) – Data matrix,
-
score
(X, y, sample_weight=None)¶ Returns the coefficient of determination R^2 of the prediction.
The coefficient R^2 is defined as (1 - u/v), where u is the residual sum of squares ((y_true - y_pred) ** 2).sum() and v is the total sum of squares ((y_true - y_true.mean()) ** 2).sum(). The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a R^2 score of 0.0.
Parameters: - X (array-like, shape = (n_samples, n_features)) – Test samples. For some estimators this may be a precomputed kernel matrix instead, shape = (n_samples, n_samples_fitted], where n_samples_fitted is the number of samples used in the fitting for the estimator.
- y (array-like, shape = (n_samples) or (n_samples, n_outputs)) – True values for X.
- sample_weight (array-like, shape = [n_samples], optional) – Sample weights.
Returns: score – R^2 of self.predict(X) wrt. y.
Return type: float
-
set_params
(**params)¶ Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form
<component>__<parameter>
so that it’s possible to update each component of a nested object.Returns: Return type: self
-
sparsity_mask
¶ A boolean mask indicating whether features are used in prediction.
-
subsample
(*args)¶ Subsample the input using this class’s subsampling scheme.
-
transform
(X)¶ Remove columns corresponding to zero-valued coefficients.
-
-
class
group_lasso.
LogisticGroupLasso
(groups, group_reg=0.05, l1_reg=0.05, n_iter=100, tol=1e-05, subsampling_scheme=None, fit_intercept=True, random_state=None)[source]¶ Sparse group lasso regularised single-class logistic regression.
This class implements the Sparse Group Lasso [1] regularisation for logistic regression with a cross entropy penalty.
This class is implemented as both a regressor and a transformation. If the
transform
method is called, then the columns of the input that correspond to zero-valued regression coefficients are dropped.The loss is optimised using the FISTA algorithm proposed in [2] with the generalised gradient-based restarting scheme proposed in [3]. This algorithm is not as accurate as a few other optimisation algorithms, but it is extremely efficient and does recover the sparsity patterns. We therefore reccomend that this class is used as a transformer to select the viable features and that the output is fed into another classification algorithm, such as LogisticRegression in scikit-learn.
References
[1] Simon, N., Friedman, J., Hastie, T., & Tibshirani, R. (2013). A sparse-group lasso. Journal of Computational and Graphical Statistics, 22(2), 231-245.
[2] Beck A, Teboulle M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences. 2009 Mar 4;2(1):183-202.
[3] O’Donoghue B, Candes E. (2015) Adaptive restart for accelerated gradient schemes. Foundations of computational mathematics. Jun 1;15(3):715-32.
-
fit_transform
(X, y, lipschitz=None)¶ Fit a group lasso model to X and y and remove unused columns from X
-
get_params
(deep=True)¶ Get parameters for this estimator.
Parameters: deep (boolean, optional) – If True, will return the parameters for this estimator and contained subobjects that are estimators. Returns: params – Parameter names mapped to their values. Return type: mapping of string to any
-
loss
(X, y)¶ The group-lasso regularised loss with the current coefficients
Parameters: - X (np.ndarray) – Data matrix,
X.shape == (num_datapoints, num_features)
- y (np.ndarray) – Target vector/matrix,
y.shape == (num_datapoints, num_targets)
, ory.shape == (num_datapoints,)
- X (np.ndarray) – Data matrix,
-
score
(X, y, sample_weight=None)¶ Returns the mean accuracy on the given test data and labels.
In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.
Parameters: - X (array-like, shape = (n_samples, n_features)) – Test samples.
- y (array-like, shape = (n_samples) or (n_samples, n_outputs)) – True labels for X.
- sample_weight (array-like, shape = [n_samples], optional) – Sample weights.
Returns: score – Mean accuracy of self.predict(X) wrt. y.
Return type: float
-
set_params
(**params)¶ Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form
<component>__<parameter>
so that it’s possible to update each component of a nested object.Returns: Return type: self
-
sparsity_mask
¶ A boolean mask indicating whether features are used in prediction.
-
subsample
(*args)¶ Subsample the input using this class’s subsampling scheme.
-
transform
(X)¶ Remove columns corresponding to zero-valued coefficients.
-
-
class
group_lasso.
MultinomialGroupLasso
(groups, group_reg=0.05, l1_reg=0.05, n_iter=100, tol=1e-05, subsampling_scheme=None, fit_intercept=True, random_state=None)[source]¶ Sparse group lasso regularised multi-class logistic regression.
This class implements the Sparse Group Lasso [1] regularisation for multinomial regression (also known as multi-class logistic regression) with a cross entropy penalty.
This class is implemented as both a regressor and a transformation. If the
transform
method is called, then the columns of the input that correspond to zero-valued regression coefficients are dropped.The loss is optimised using the FISTA algorithm proposed in [2] with the generalised gradient-based restarting scheme proposed in [3]. This algorithm is not as accurate as a few other optimisation algorithms, but it is extremely efficient and does recover the sparsity patterns. We therefore reccomend that this class is used as a transformer to select the viable features and that the output is fed into another classification algorithm, such as LogisticRegression in scikit-learn.
References
[1] Simon, N., Friedman, J., Hastie, T., & Tibshirani, R. (2013). A sparse-group lasso. Journal of Computational and Graphical Statistics, 22(2), 231-245.
[2] Beck A, Teboulle M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences. 2009 Mar 4;2(1):183-202.
[3] O’Donoghue B, Candes E. (2015) Adaptive restart for accelerated gradient schemes. Foundations of computational mathematics. Jun 1;15(3):715-32
-
fit
(X, y, lipschitz=None)¶ Fit a group-lasso regularised linear model.
-
fit_transform
(X, y, lipschitz=None)¶ Fit a group lasso model to X and y and remove unused columns from X
-
get_params
(deep=True)¶ Get parameters for this estimator.
Parameters: deep (boolean, optional) – If True, will return the parameters for this estimator and contained subobjects that are estimators. Returns: params – Parameter names mapped to their values. Return type: mapping of string to any
-
loss
(X, y)¶ The group-lasso regularised loss with the current coefficients
Parameters: - X (np.ndarray) – Data matrix,
X.shape == (num_datapoints, num_features)
- y (np.ndarray) – Target vector/matrix,
y.shape == (num_datapoints, num_targets)
, ory.shape == (num_datapoints,)
- X (np.ndarray) – Data matrix,
-
score
(X, y, sample_weight=None)¶ Returns the mean accuracy on the given test data and labels.
In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.
Parameters: - X (array-like, shape = (n_samples, n_features)) – Test samples.
- y (array-like, shape = (n_samples) or (n_samples, n_outputs)) – True labels for X.
- sample_weight (array-like, shape = [n_samples], optional) – Sample weights.
Returns: score – Mean accuracy of self.predict(X) wrt. y.
Return type: float
-
set_params
(**params)¶ Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form
<component>__<parameter>
so that it’s possible to update each component of a nested object.Returns: Return type: self
-
sparsity_mask
¶ A boolean mask indicating whether features are used in prediction.
-
subsample
(*args)¶ Subsample the input using this class’s subsampling scheme.
-
transform
(X)¶ Remove columns corresponding to zero-valued coefficients.
-