[pystatsmodels] Warning again: GLM has more options than unit test cases

[j***@gmail.com]

https://github.com/statsmodels/statsmodels/issues/5316
GLM has all the historical test for IRLS. It looks like gradient
optimization in GLM still does not have enough test coverage to catch
refactoring mistakes.

I found this because my standard errors didn't match in the penalized case
while looking at gam and similar L2 penalized GLM.
https://github.com/statsmodels/statsmodels/pull/5296

Scale handling in GLM is a nasty (complex and fragile) business.
Many of the unit test that I added in last few years are for fixed scale or
for sandwiches which don't use scale directly for cov_params. The heavily
unit tested cases are GLM Logit and Poisson which both have fixed scale,
and not many unit test for the QMLE excess dispersion case.

The offending refactoring bug has not been released, i.e. it happened after
0.9, and can be fixed before the next release.
https://github.com/statsmodels/statsmodels/pull/4620/commits/407807912a79c24eed09a39c991b07a40015e8f5
(a reason not to have very short release cycles)

Josef

[Brock Mendel]

It looks like gradient optimization in GLM still does not have enough

test coverage to catch refactoring mistakes.

I would be ecstatic to fix this. #4936 was explicitly opened to improve
the testing for IRLS.

This is one of many test-improvement PRs ready and waiting.

Carthago Delenda Est.

https://github.com/statsmodels/statsmodels/issues/5316
GLM has all the historical test for IRLS. It looks like gradient
optimization in GLM still does not have enough test coverage to catch
refactoring mistakes.
I found this because my standard errors didn't match in the penalized case
while looking at gam and similar L2 penalized GLM.
https://github.com/statsmodels/statsmodels/pull/5296
Scale handling in GLM is a nasty (complex and fragile) business.
Many of the unit test that I added in last few years are for fixed scale
or for sandwiches which don't use scale directly for cov_params. The
heavily unit tested cases are GLM Logit and Poisson which both have fixed
scale, and not many unit test for the QMLE excess dispersion case.
The offending refactoring bug has not been released, i.e. it happened
after 0.9, and can be fixed before the next release.
https://github.com/statsmodels/statsmodels/pull/4620/commits/407807912a79c24eed09a39c991b07a40015e8f5
(a reason not to have very short release cycles)
Josef

[j***@gmail.com]

Post by Brock Mendel

It looks like gradient optimization in GLM still does not have enough

test coverage to catch refactoring mistakes.
I would be ecstatic to fix this. #4936 was explicitly opened to improve
the testing for IRLS.
This is one of many test-improvement PRs ready and waiting.

That's a lot of reshuffling but didn't add any new unit tests or asserts.

It looks like a useful streamlining but still requires time for review, and
I had not been working in that neighborhood for a while.
I prefer to review or work on this in the context of specific extensions.

At the moment I'm a bit annoyed that I have to special case GLM quite a bit
for generic penalization, and had to do it also for other generic meta
methods.
GLM is still a model that cannot make up it's mind whether it is a
(generalized) linear irls model or a standard nonlinear MLE/QMLE model.
It has been the former for a long time but it's moving towards being the
latter.
On the other hand, GLM is one of my favorite playgrounds because any
extension immediately becomes applicable to a large set of use cases, and
is useful to build prototypes of generic extensions.

Josef

Post by Brock Mendel
Carthago Delenda Est.

https://github.com/statsmodels/statsmodels/issues/5316
GLM has all the historical test for IRLS. It looks like gradient
optimization in GLM still does not have enough test coverage to catch
refactoring mistakes.
I found this because my standard errors didn't match in the penalized
case while looking at gam and similar L2 penalized GLM.
https://github.com/statsmodels/statsmodels/pull/5296
Scale handling in GLM is a nasty (complex and fragile) business.
Many of the unit test that I added in last few years are for fixed scale
or for sandwiches which don't use scale directly for cov_params. The
heavily unit tested cases are GLM Logit and Poisson which both have fixed
scale, and not many unit test for the QMLE excess dispersion case.
The offending refactoring bug has not been released, i.e. it happened
after 0.9, and can be fixed before the next release.
https://github.com/statsmodels/statsmodels/pull/4620/commits/407807912a79c24eed09a39c991b07a40015e8f5
(a reason not to have very short release cycles)
Josef