scikit-learn: GLM doesn't have an offset option

Describe the workflow you want to enable

As described at #24155, GLM should also support the offset option.

Describe your proposed solution

For example

from sklearn.linear_model import PoissonRegressor

glm = PoissonRegressor(
                        alpha=0,
                        fit_intercept=False, 
                        max_iter=300,
                        offset=A # offset option
                      )
glm.fit(X_train, y_train)

Describe alternatives you’ve considered, if relevant

No response

Additional context

No response

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Reactions: 1
  • Comments: 15 (11 by maintainers)

Most upvoted comments

Hi all, are there any news on this feature? Do you think that there are any possibilities that it will be implemented in a near future? 😊

In the GLM literatur, offsets have a clear meaning. This is reflected by implementations of GLMs in R where offset can be part of the model formula:

## an example with offsets from Venables & Ripley (2002, p.189)
utils::data(anorexia, package = "MASS")

anorex.1 <- glm(Postwt ~ Prewt + Treat + offset(Prewt),
                family = gaussian, data = anorexia)
summary(anorex.1)

statsmodels, Julia.

My main concern is that it does not fit in our API.

For reference, glum uses the 3rd approach, i.e. argument in fit and predict, see https://glum.readthedocs.io/en/latest/glm.html#glum.GeneralizedLinearRegressor.fit.

Hi, with @manudarmi and our team we are working on adding offsets to the GLMs.

We have a working implementation which passes the offset as a kwarg of the fit method.

It would look like this:

from sklearn.linear_model import GammaRegressor

reg = GammaRegressor(...)
reg.fit(X, y, sample_weight=sample_weight, offset=offset)

With offset being an array-like with the same shape as sample_weight.

The main problem is that whenever offsets are not known a priori (e.g. computed during a previous step in a pipeline), it would be impossible to pass them to the GLM without creating a custom wrapper.

For example:

from sklearn.linear_model import GammaRegressor
from sklearn.pipeline import Pipeline

pipeline = Pipeline([
    ("create_the_offsets", CustomTransformation(...)),
    ("regressor": GammaRegressor(...)),
])
# no way to pass the offset from `CustomTransformation` output into the GLM

A solution could be to track whether the GLM expects offsets or not as a constructor argument, and then expect them to be either in the first or last column of X. This would make the snippet above work properly, at the cost of being less explicit.

Would look like this:

import numpy
from sklearn.linear_model import GammaRegressor

X = numpy.concatenate([X, offset], axis=1) # offset in the last column
reg = GammaRegressor(use_offset=True, **other_params)
reg.fit(X, y, sample_weight=sample_weight)

If you could give me some feedback I could start working towards a PR in either direction 😄

it’s not obvious to me what are the use cases for such an option

I know about 2 use cases:

  1. It’s a way to deal with “exposure”. But this can also (or better) be accomplished with sample weights (model y=count/w with sample weight w, same is true for logistic regression).
  2. One may have a priori knowledge of the effect of a certain feature or certain features, or one wants to fix it (for other reasons).

besides manually constraining the intercept

It’s usually not the intercept, but a certain (amount of) coefficient(s) that one wants to fix.