How to improve Forecasting Quality for Demand Forecasting

This article explains how to improve forecasting quality for the Demand Forecasting ML Solution Template in the AI & Analytics Engine

Forecasting Quality in Demand Forecasting

Forecasting quality is used to determine a forecast model’s performance in the AI & Analytics Engine. Having a good forecasting quality is essential for a demand forecasting model as this is a good indicator that future forecasts made from this model will be accurate.

However, there can be applications that don't give a satisfactory forecasting quality initially. This article describes some approaches to potentially improve forecasting quality when that happens.

Change model training

The forecasting quality can depend on the model. Therefore, identifying the model that best fits your forecasting application is essential. The Engine allows you to train different models from an extensive list of available algorithms. Typically models like Random Forest, XGBoost, LightGBM, and Gradient Boosting perform well for forecasting problems.

To change the model and their configurations in a ready app, use the Train new model option in the Model Training tile in the app summary. This leads to the Select algorithms dialogue where you can select algorithms to train from many standard algorithms.

Train new model from the App summary page

Select algorithms

Change data and configurations

Sometimes changing the algorithm may have no effect, especially if the issue of poor forecast quality arises from the richness and relevance of the data. In such cases, changing data or app configurations can be necessary.

To do this, you can use the option Create another app using this setup in the Engine. Once clicked, this leads you to the app builder page for a new app with inputs already pre-filled from the original app. You can modify these inputs based on some of the below recommendations and build the app.

Click Create another app using this setup

Use more demand data

There can be cases where the time span of the historical data is too short to identify patterns in demand that can lead to good forecasts in the future. In such cases, the simplest approach is to include more data going back further in history, thus increasing the time span of the historical data.

For example, you may have tried to build an app with the most recent 2 years of data and you might instead try using the most recent 5 years.

Add more demand data in the new app builder

Use additional optional datasets

The Engine allows adding supporting data that can potentially improve model performance. For example, for the retail use case in demand forecasting, you can provide data about products and locations where sales were made. Holiday data can also be a good indicator of the demand. If you already have such supporting data that hasn’t been used, the Engine allows you to provide it.

Add product info data

Add location info data

Add holiday data

Change seasonality configurations

Demand typically exhibits seasonal patterns. Knowing these seasonal patterns can be extremely useful in forecasting demand. Providing the most appropriate seasonal pattern configurations during the app building stage can help forecasting models to perform better.

If you already provided the seasonal configurations but the model needs improvement, then try modifying them. If not, try adding new seasonal patterns based on your knowledge of the business and domain.

Change seasonality configurations

Change the amount of relevant past data

During the app building stage, you can specify how far back into the past the data is still relevant for forecasting.

On one hand, if this time period is too short, the forecasting model may not have enough data to identify patterns that lead to better forecasts. On the other hand, if this period is too long and extreme past data were to be used, the model may be trained on patterns or behaviours which are out of date, resulting in poor model performance.

Therefore, it is important to pick this time period in the most appropriate way, based on your domain and business knowledge.

Change the amount of relevant past data

Remove groups with poor performance

There can be some groups to which model forecasts are not good. This can be due to lack of data for that group or due to inaccurate past data, etc. Identify such groups and remove them from the data if possible. Or you can also bundle multiple small and insignificant groups into a single category offline.

Modify business requirements

If you have the flexibility to modify business requirements such as granularity of the forecasting horizon, forecast period or the lead time, then try that.

Typically, shorter forecast periods lead to better model performance. So does the shorter lead times. In both cases, the model can focus on more recent data during forecasting leading to better forecasting quality.

Modify business requirements

Modify hierarchical granularity levels

Also if possible modify hierarchical granularity levels. This may result in removing non-essential and undeperforming granularity levels from the forecasts leading to better performance.

Modify hierarchical granularity levels in the new app builder