How can you best leverage the benefits of statistical forecasting models when forecasting?

Time-series and causal models are typically used when forecasting. However, they have their limitations:

1. Inaccurate for long-term forecasting
   If you forecast on a long horizon, they can prove to be inaccurate. Time-series models are based on the assumption that past patterns will continue into the future. However, this is not always the case, especially for long-term forecasting. If there is a sudden change in the market, a time-series model may not be the best in accurately predicting future demand.
2. Sensitive to outliers
   Extreme values or anomalies in the time series can distort the model's predictions, especially if they are not appropriately handled or detected.
3. Inability to capture evolving relationship between variables
   Causal relationships between variables may evolve over time, and these changes may not be captured adequately by static causal models, compromising accuracy. Here’s an example - we’ve had customers that consistently used GDP as an indicator to watch, and typically use correlation to, correlate the relationship with their main variable. 
   
   But something happens three to four months down the road. They find that GDP, in this case, no longer has a proportional relationship to their main variable; their main variable is no longer heading in the same direction. This gets compounded if GDP then is also correlated with additional variables.
   
4. Limitations of causation
   There is also an assumption of linearity. Some causal models assume linear relationships between variables, which may not hold true in real-world scenarios with more complex and nonlinear relationships.

"The use of model averaging and weighted forecast results based on accuracy instill confidence in the forecasts. When organizations have a clearer understanding of the accuracy levels of each model, they can place more trust in the final forecast and make more confident decisions.

How to optimize your forecast performance

If you are currently using these forecasting models in your forecasting, here's what you should consider to ensure you’re optimizing your forecast performance:

To begin with, what are these search methodologies and what do they do best?

Firstly, use multiple forecasting models

By using multiple forecasting models and comparing the results, you can minimize the risk of making poor decisions based on a single forecast. 

This can arguably be time-consuming.

It can take anywhere from a few hours to a few weeks to build and test multiple univariate and multivariate forecasting models. To compound that, the time taken will depend on the complexity of the models. More complex models will invariably take longer to build. If you data happens to not be clean (outliers, seasonality), it can also extend the time taken to build the models.

Indicio cleans your data in seconds, builds & backtests 30+ models, and performs model averaging, which significantly speeds up the analysis. All this can quickly give you an idea of:

1. Which market drivers you need to pay attention to when forecasting

2. Presents you with the top three most accurate models for forecasting your main variable (Eg: future demand or sales)

3. Get the weighted forecast result, together with a confidence interval for the prediction. [The weights are chosen according to the forecast accuracy of each model).

The use of model averaging and weighted forecast results based on accuracy instill confidence in the forecasts. When organizations have a clearer understanding of the accuracy levels of each model, they can place more trust in the final forecast and make more confident decisions.

[Here's how it's done]

‍What’s the business impact?  
Having the capacity to identify the key market drivers that significantly impact your main variable can guide you in adapting your strategies to changing market conditions.

Being able to get all these steps done in a short period of time with a forecasting tool saves you time and resources. Time that can be spent on interpreting the results and making decisions instead of getting bogged down in the model creation process.

Next, monitor your forecast performance

‍Another area to take into consideration is ensuring you monitor your forecasts, and make adjustments as needed. This is especially important if there are changes in the factors that affect demand as you might need to relook the relevancy of your leading indicators.

You’d perhaps want to introduce and test the impact of new market drivers on your main variable.

By comparing your forecasts to actual results on a regular basis, you can track the accuracy of your forecasts, identify any new relevant indicators. and make the necessary adjustments.  

‍How do you monitor your forecast performance?
‍
‍A variety of metrics can be used to measure accuracy, such as the mean absolute percentage error (MAPE) or the root mean squared error (RMSE). but there are others as well. Using a variety of metrics can give you a more complete picture of your forecast performance.

In Indicio:

- You get an overview of your previous forecasts, which you can then compare against your actual data to backtest and evaluate model performance.  

- You get a breakdown of each metric’s performance; MAPE, MAE, RMSE.
‍
[Find out how a leader in the manufacturing industry improved their forecast accuracy by more than 50%]

- By using Lasso and other regularisation techniques through cross-validation to build these models, the results that Indicio generates will give you a clear and immediate picture of which indicators to pay attention to. 

‍What’s the business impact?
‍Easy verification means you not only enhance the reliability of the forecasting models used, but this also ensures you’re always factoring in the most relevant market drivers. A win-win! 

[Try it out in Indicio]