Module Ridge¶
operalib.ridge
implements Operator-Valued Kernel ridge
regression.
-
class
operalib.ridge.
Ridge
(kernel='DGauss', lbda=1e-05, A=None, gamma=1.0, theta=0.7, period='autocorr', autocorr_params=None, solver=<function fmin_l_bfgs_b>, solver_params=None, kernel_params=None)¶ Operator-Valued kernel ridge regression.
Operator-Valued kernel ridge regression (OVKRR) combines ridge regression (linear least squares with l2-norm regularization) with the (OV)kernel trick. It thus learns a linear function in the space induced by the respective kernel and the data. For non-linear kernels, this corresponds to a non-linear function in the original space.
The form of the model learned by OVKRR is identical to support vector regression (SVR). However, different loss functions are used: OVKRR uses squared error loss while support vector regression uses epsilon-insensitive loss, both combined with l2 regularization. In contrast to SVR, fitting a OVKRR model can be done in closed-form and is typically faster for medium-sized datasets. On the other hand, the learned model is non-sparse and thus slower than SVR, which learns a sparse model for epsilon > 0, at prediction-time.
See also
sklearn.Ridge
- Linear ridge regression.
sklearn.KernelRidge
- Kernel ridge regression.
sklearn.SVR
- Support Vector Regression implemented using libsvm.
References
- Kevin P. Murphy “Machine Learning: A Probabilistic Perspective”, The MIT Press chapter 14.4.3, pp. 492-493
Examples
>>> import operalib as ovk >>> import numpy as np >>> n_samples, n_features, n_targets = 10, 5, 5 >>> rng = np.random.RandomState(0) >>> y = rng.randn(n_samples, n_targets) >>> X = rng.randn(n_samples, n_features) >>> clf = ovk.Ridge('DGauss', lbda=1.0) >>> clf.fit(X, y)
Attributes
dual_coef_ (array, shape = [n_features] or [n_targets, n_features]) Weight vector(s) in kernel space self.linop_ (callable) Callable which associate to the training points X the Gram matrix (the Gram matrix being a LinearOperator) A_ (array, shape = [n_targets, n_targets]) Set when Linear operator used by the decomposable kernel is default or None. period_ (float) Set when period used by the First periodic kernel is ‘autocorr’. solver_res_ (any) Raw results returned by the solver. Methods
fit
(X, y)Fit OVK ridge regression model. get_params
([deep])Get parameters for this estimator. predict
(X)Predict using the OVK ridge model. score
(X, y[, sample_weight])Returns the coefficient of determination R^2 of the prediction. set_params
(**params)Set the parameters of this estimator. -
__init__
(kernel='DGauss', lbda=1e-05, A=None, gamma=1.0, theta=0.7, period='autocorr', autocorr_params=None, solver=<function fmin_l_bfgs_b>, solver_params=None, kernel_params=None)¶ Initialize OVK ridge regression model.
Parameters: kernel : {string, callable}, default=’DGauss’
Kernel mapping used internally. A callable should accept two arguments and the keyword arguments passed to this object as kernel_params, and should return a LinearOperator.
lbda : {float}, default=1e-5
Small positive values of lbda improve the conditioning of the problem and reduce the variance of the estimates. Lbda corresponds to
(2*C)^-1
in other linear models such as LogisticRegression or LinearSVC.A : {LinearOperator, array-like, sparse matrix}, default=None
Linear operator used by the decomposable kernel. If default is None, wich is set to identity matrix of size y.shape[1] when fitting.
gamma : {float}, default=1.
Sigma parameter for the Decomposable Gaussian kernel. Ignored by other kernels.
theta : {float}, default=.7
Theta parameter for the Decomposable First Periodic kernel. Ignored by other kernels.
period : {float}, default=default_period
Period parameter for the First periodic kernel. If optional modules have been imported then default_period is 2 * pi. Otherwise it uses autocorrelation methods to determine the period.
solver : {callable}, default=scipy.optimize.fmin_l_bfgs_b
Solver able to find the minimum of the ridge problem. scipy.optimize.fmin_l_bfgs_b(*solver_params)[0] must return the optimal solution.
autocorr_params : {mapping of string to any}
Additional parameters (keyword arguments) for the period detection for periodic kernels. If None, parameter choice is left to the period detection method.
solver_params : {mapping of string to any}, optional
Additional parameters (keyword arguments) for solver function passed as callable object.
kernel_params : {mapping of string to any}, optional
Additional parameters (keyword arguments) for kernel function passed as callable object.
-
fit
(X, y)¶ Fit OVK ridge regression model.
Parameters: X : {array-like, sparse matrix}, shape = [n_samples, n_features]
Training data.
y : {array-like}, shape = [n_samples] or [n_samples, n_targets]
Target values.
Returns: self : returns an instance of self.
-
predict
(X)¶ Predict using the OVK ridge model.
Parameters: X : {array-like, sparse matrix}, shape = [n_samples, n_features]
Samples.
Returns: C : {array}, shape = [n_samples] or [n_samples, n_targets]
Returns predicted values.