Out-of-sample evaluation

[sehoff]

I am sorry to ask this probably naive question, but I need to be sure and I am not coding in R often.

• When using cv.BigVAR() the right-hand side endogenous variables are automatically lagged?
• results@preds in the example below are oos predictions, i.e., the first prediction is obtained from an estimation up to T_2 for T_2 +1. The next predictions are rolled over by one period, in my case, keeping the optimal lambda fixed?
• I need to lag exogenous variables myself, such that the timing corresponds to equation (1) in the vignette.

Here is my code, which should give for each of the endogenous variables in the system a separate OOS R2 (benchmarked wrt to conditional mean forecast):

VARX=list(k=5,s=1)
mod1<-constructModel(y_full,
                    VARX=VARX,
                    p=1,
                    "BasicEN",
                    gran=c(100,10),
                    h=1,
                    cv="Rolling",
                    verbose=FALSE,
                    IC=TRUE,
                    model.controls=list(intercept=TRUE, alpha=0.5))

results=cv.BigVAR(mod1)

model.pred <- results@preds
mean.pred  <- results@MeanPreds
# test dep var
y.test     <- y[floor(2/3*nrow(y_full)+1):nrow(y_full),]
# MSFE
msfe.model <- colMeans((y.test - model.pred)**2)
msfe.mean  <- colMeans((y.test - mean.pred)**2)
# OOS R2
1- msfe.model/msfe.mean

[wbnicholson]

When using cv.BigVAR() the right-hand side endogenous variables are automatically lagged?

Yes, the both endogenous and exogenous variables are lagged automatically according to p and s in constructModel.

results@preds in the example below are oos predictions, i.e., the first prediction is obtained from an estimation up to T_2 for T_2 +1. The next predictions are rolled over by one period, in my case, keeping the optimal lambda fixed?

Yes, that's correct.

I need to lag exogenous variables myself, such that the timing corresponds to equation (1) in the vignette.

This is done automatically by BigVAR. If you have a non-standard lag order, you can lag it yourself and set s=0 and tf=TRUE which will not perform and lagging.

Your code looks correct. Let me know if you have additional questions.

[sehoff]

Thanks a lot for the quick reply!