For iterative solvers, it is possible to provide a warm-start for the process via an initial guess for the coefficients (for example, these may have been obtained via previous fit). When provided with too many coefficients, only the first \(k\) coefficients will be used, where \(k\) is the expected number of coefficients. The intercept will always be the last member of the provided array. If too few coefficients are provided, the initial guess will be ignored.
Note
For the conjugate gradient solver in the underdetermined case (\(n_{\mathrm{features}} > n_{\mathrm{samples}}\)) the dual problem is solved, and the initial coefficients provided should be the dual ones (the expected number of coefficients \(k\) is equal to \(n_{\mathrm{samples}}\) instead of \(n_{\mathrm{features}}\)).
Initial coefficients can be provided in the following way:
Provide x0 parameter when calling aoclda.linear_model.linmod.fit().
Set warm_start option to True in aoclda.linear_model.linmod(). This will turn on automatic reuse of coefficients from the previous aoclda.linear_model.linmod.fit() call.
Note that when the expected number of coefficients changes (due to different number of columns in dataset), we will either truncate previous coefficients or pad them with 0 to match the expected size.
For example, when data initially had 3 features and we fit model with intercept, the layout of coefficients looks like: \([\mathrm{feat}_1, \mathrm{feat}_2, \mathrm{feat}_3, \mathrm{intercept}]\).
Now, using warm_start with new dataset that has 4 features will result in supplying initial coefficients: \([\mathrm{feat}_1, \mathrm{feat}_2, \mathrm{feat}_3, 0.0, \mathrm{intercept}]\).
Conversely, supplying with data with 2 features, will reuse only first 2 coefficients from previous call: \([\mathrm{feat}_1, \mathrm{feat}_2, \mathrm{intercept}]\).
Analogously, when coefficients are a matrix (multi-class Logistic Regression), it truncates or pads with zeros the entire column(s) of the coefficient matrix.
Warning
There are following limitations to reusing coefficients from previous fit:
In multi-class Logistic Regression, if number of classes has changed in comparison to previous fit.
If Conjugate Gradient method has been used to first fit well-determined data, and then it is attempted to fit it on under-determined data.
Provide coefs pointer to initial coefficients while calling da_linmod_fit_start_?.