[pystatsmodels] n step ahead forecasts for unobserved components model

[Alastair Heggie]

I have an unobserved components model.

I can specify the steps argument for the forecast method to get 1 to n step
ahead forecasts from the end of the data. I can also use the
get_predictions method to get the one step ahead predictions at all points
in the data.

However, as far as I can see the methods don't allow me to obtain the 1 to
n step ahead forecasts at all time points in the endogenous data. The
result I want would be an [n_observations x n] matrix of forecasts.

Am I correct that the default forecasting methods don't support what I want
to do?

I have found one way to get what I want but it seems very inefficient, I am
hoping someone can suggest a better way:

I can fit the model to obtain the variance parameters. I can then filter
the model n_obs times to the first 1, 2, 3...n_obs data points and eac time
use the forecast method to get the 1-n step ahead forecast.

To avoid having to run the Kalman filter multiple times I have tried to
create a new model for every data point and apply the filtering with the
state initialised using the state estimates from the previous model using
the initialize_known method. However the forecasts obtained from this
method are not correct. As far as I can see the initialize_known method
does not seem to work because nothing changes about the forecasts if I omit
the initialize_known step.

Best wishes,
Alastair

[Chad Fulton]

Post by Alastair Heggie
I have an unobserved components model.
I can specify the steps argument for the forecast method to get 1 to n
step ahead forecasts from the end of the data. I can also use the
get_predictions method to get the one step ahead predictions at all points
in the data.
However, as far as I can see the methods don't allow me to obtain the 1 to
n step ahead forecasts at all time points in the endogenous data. The
result I want would be an [n_observations x n] matrix of forecasts.
Am I correct that the default forecasting methods don't support what I
want to do?
I have found one way to get what I want but it seems very inefficient, I
I can fit the model to obtain the variance parameters. I can then filter
the model n_obs times to the first 1, 2, 3...n_obs data points and eac time
use the forecast method to get the 1-n step ahead forecast.
To avoid having to run the Kalman filter multiple times I have tried to
create a new model for every data point and apply the filtering with the
state initialised using the state estimates from the previous model using
the initialize_known method. However the forecasts obtained from this
method are not correct. As far as I can see the initialize_known method
does not seem to work because nothing changes about the forecasts if I omit
the initialize_known step.
Best wishes,
Alastair

The most straightforward solution is to use dynamic prediction, as long as
you want to use the parameters estimated using the whole sample to create
your forecasts at each time point.

Basically, that looks like:

```
pred1 = np.zeros((n_forecasts, mod.nobs))

for t in range(mod.nobs):
pred1[:, t] = res.predict(start=t, end=t + n_forecasts - 1,
dynamic=True)
```

But you can make it work the other two ways you suggested, but running all
or part of the the Kalman filter each time. I've shown how to make each of
these three methods work in this notebook:

https://gist.github.com/ChadFulton/a263918798210e354917e65a17679a24

Best,
Chad

[Alastair Heggie]

Thanks for this and for going to the trouble of making the demonstration
notebook. The dynamic prediction method achieves what I want to do.
Interesting to see what I was getting wrong with initialize_known.

Post by Chad Fulton

Post by Alastair Heggie
I have an unobserved components model.
I can specify the steps argument for the forecast method to get 1 to n
step ahead forecasts from the end of the data. I can also use the
get_predictions method to get the one step ahead predictions at all points
in the data.
However, as far as I can see the methods don't allow me to obtain the 1
to n step ahead forecasts at all time points in the endogenous data. The
result I want would be an [n_observations x n] matrix of forecasts.
Am I correct that the default forecasting methods don't support what I
want to do?
I have found one way to get what I want but it seems very inefficient, I
I can fit the model to obtain the variance parameters. I can then filter
the model n_obs times to the first 1, 2, 3...n_obs data points and eac time
use the forecast method to get the 1-n step ahead forecast.
To avoid having to run the Kalman filter multiple times I have tried to
create a new model for every data point and apply the filtering with the
state initialised using the state estimates from the previous model using
the initialize_known method. However the forecasts obtained from this
method are not correct. As far as I can see the initialize_known method
does not seem to work because nothing changes about the forecasts if I omit
the initialize_known step.
Best wishes,
Alastair

The most straightforward solution is to use dynamic prediction, as long as
you want to use the parameters estimated using the whole sample to create
your forecasts at each time point.
```
pred1 = np.zeros((n_forecasts, mod.nobs))
pred1[:, t] = res.predict(start=t, end=t + n_forecasts - 1,
dynamic=True)
```
But you can make it work the other two ways you suggested, but running all
or part of the the Kalman filter each time. I've shown how to make each of
https://gist.github.com/ChadFulton/a263918798210e354917e65a17679a24
Best,
Chad