Criteria for a Good Forecasting Method: Factors to Keep in Mind

Understanding Forecasting Methods

Forecasting methods are essential tools for businesses and organizations to make informed decisions about the future. These methods involve using historical data and statistical models to predict future events or conditions. By analyzing past trends and patterns, businesses can anticipate future demand, optimize production planning, and make strategic decisions. There are various types of forecasting techniques, including qualitative and quantitative methods, each suited to different scenarios and data availability.

Types of Forecasting Techniques

Forecasting techniques can be broadly classified into two categories: qualitative and quantitative methods. Qualitative methods rely on expert opinions and market insights, while quantitative methods rely on numerical data and statistical models. Some common types of forecasting techniques include:

• Time Series Analysis: This method involves analyzing historical data points to identify patterns or trends over time. It is particularly useful for short-term forecasts and can help businesses understand seasonal variations and long-term trends.
• Regression Analysis: This method assesses the relationship between a dependent variable and one or more independent variables. It is useful for understanding how different factors influence demand and can help in making accurate forecasts.
• Econometric Models: These models use statistical methods to analyze economic data and test economic theories. They are often used for long-term forecasts and can incorporate various economic factors to predict future outcomes.
• Delphi Method: This method involves gathering expert opinions and using them to make predictions about future events. It is particularly useful when historical data is limited or when forecasting new products or markets.
• Market Research: This method involves gathering data from customers, industry experts, and other stakeholders to make predictions about future market trends. It provides valuable insights into consumer behavior and market demand.

Factors To Consideration in The Choice of a Demand Forecasting Method

A famous quote coined by Paul Saffo, a consultant associate professor at Stanford University who has been a technology forecaster for more than 35 years now, says that- “The goal of forecasting is not to predict the future but to tell you what you need to know to take meaningful action in the present”. One of the sole reasons for the survival of the _Homo Sapiens – the ONLY surviving human species, _is that they had the ability to prepare themselves in advance, thus safe-guarding themselves from any adverse conditions of the future. Ensuring smooth functioning in any setting – be it a simple household or a high functioning business or even the running of a country, involves forecasting several events which are likely to happen in the future and strengthening the course of action. This probably justifies the immense research and efforts being put into improving the forecasting techniques all around the world.

A good forecasting method has many characteristics, the most important one being accuracy. Inaccurate forecasts can cause a lot of damage sending any system into overdrive or an undesirable inactivity. In addition to accuracy, forecasts should be up-to-date, timely, reliable and plausible.

Broadly speaking, there are three types of forecasting techniques –

Nature of Data

The nature and quality of available data significantly influence the selection of the forecasting technique. If historical data is available, quantitative methods such as time series analysis or causal models are suitable. These methods leverage historical sales data and other relevant metrics to predict future demand accurately. When data is limited or unavailable, qualitative methods like the Delphi technique or market research may be used. These methods rely on expert opinions and market insights to make informed predictions.

Time Horizon

The time horizon of the forecasting technique plays a crucial role in determining the approach. Short-term forecasts (up to one year) focus on immediate trends and patterns, using time series analysis to capture seasonal variations and short-term fluctuations. Long-term forecasts (beyond one year) often use causal models to account for broader economic factors and long-term trends. Understanding the appropriate time horizon helps businesses choose the right forecasting method and make more accurate predictions.

Quantitative Forecasting:

Quantitative forecasting techniques are primarily used in the early stages of a set-up when the amount of data available is less. For example, determining the demand of a new product would require this methodology since no historical data, which is essential for quantitative modeling, will be at hand. These techniques include market research, combined expert opinions in a panel, studying the growth of similar products in the market, simulating similar conditions for the new commodity, and finally the Delphi method (a questionnaire-based approach in which a series of experts are made to answer various questions whose answers determine the questions of the next questionnaire, enabling maximum sharing of information for effective forecasting amongst this group of experts).

Time Series Forecasting: Using Historical Data

When a large amount of historical data is available, a time series analysis is a good forecasting technique to employ. Accurate demand forecasts are crucial for making informed decisions about production policies, financial requirements, and resource allocation. Using this analysis one can study trends, seasonalities and the randomness in the life cycle of an entity. These methods include moving averages, exponential smoothing, trend projections, ARIMA models and so on.

While using this methodology, it is important to use a stationary time series. Any chronologically ordered data which is centered about a constant mean,have a constant variance and be covariance stationary. In other words, the above mentioned statistics should not be a function of (or change with) time for the given time series. The following image shows non-stationarity in terms of mean, variance and covariance respectively for the plots in the red colour.

Stationarity is usually absent in real life data, so many methods are incorporated to achieve a somewhat stationary time series – because it becomes very difficult to model non-stationarity. Using logarithms or differencing techniques helps to make a time series stationary. These transformations being fairly simple, help model situations easily and help to obtain the forecasts in their original format.

It is also a good idea to check the normality of the series and power transform to make the data normal, if required. This can help improve the performance of a time series model since normality is an underlying assumption for most statistical models.

Another factor to take into account while using time series forecasting is the horizon for which a forecast is being generated. Short-period (< = 1 year) forecasts are more accurate than long-term (5-10 years) or very long period (>10 years) forecasts.

The level of forecasting can also affect the nature of the forecasts. Typically there are 3 levels to be considered – the micro level, the industry level and the macro level. Consider a firm which sells shoes which has stores across India. The different levels of forecasts here can be at a store level, a city level or a state level. It is obvious that the forecasts obtained at each level will be different and will require a different level of accuracy at each level to function correctly.

Finally, retraining of a model is crucial for every new prediction since the model needs to capture the latest changes in the behaviour of the entity for which the forecast is being generated.

Causal Modeling: Regression Analysis

Demand forecasting is probably the most sophisticated of the forecasting techniques that exist and they capture relationships among the variables. These techniques include regression models, econometric models, economic input-output models etc.

While applying this technique, it is important to take care of the underlying assumptions to obtain the best forecasts. Take a simple regression analysis for example. To get accurate forecasts one must ensure normality of the independent variables. There should be no relationship amongst the dependent variables (no multicollinearity), the variables must be homoscedastic and there should be no autocorrelation present.

Choosing the right type of technique is essential to improve the quality of the forecasts.

Evaluating Forecast Performance

Evaluating forecast performance is essential to determine the accuracy and reliability of the forecasting method. By assessing how well a forecast predicts actual outcomes, businesses can refine their forecasting techniques and improve future predictions. There are various criteria used to evaluate forecast performance, ensuring that the chosen method provides accurate and actionable insights.

Forecast Performance Evaluation Criteria

• Mean Absolute Deviation (MAD): This measure calculates the average magnitude of errors between actual and forecast data. It provides a straightforward way to assess the accuracy of a forecast.
• Percent of Accuracy (POA): This measure calculates the ratio of forecast sales to actual sales during the holdout period. It helps businesses understand how close their forecasts are to actual performance.
• Root Mean Square Error (RMSE): This measure calculates the square root of the average of the squared differences between actual and forecast data. It is particularly useful for identifying large errors and understanding the overall accuracy of a forecast.
• Mean Absolute Percentage Error (MAPE): This measure calculates the average absolute percentage difference between actual and forecast data. It provides a normalized measure of forecast accuracy, making it easier to compare across different datasets.

By using these criteria, businesses and organizations can evaluate the performance of their forecasting methods and make informed decisions about the future. Accurate forecasts are crucial for effective production planning, inventory management, and strategic decision-making, ensuring that businesses can meet market demand and achieve their goals.

Uncertainty as a factor

Harvard Business Review also talks about the ability of a forecast to capture uncertainty as an extremely important factor to keep in mind while forecasting. While considering the features of a model, a forecaster needs to take into account those variables which are somewhat improbable but not impossible to encapsulate uncertainty.

It is first important to describe what is the level of uncertainty one wishes to capture. Suppose that a researcher is trying to create a sales forecast for shoes in a particular store. The various features which can influence this directly are – the geographical area of the store (say city or village), the price range of the product (affects the type of customers it attracts), types of shoes it sells (sports or casual), capacity of the store etc. However if the forecaster wants to prepare for future shutdowns due to natural calamities or terrorist attacks which might not be very probable (but not impossible, no one imagined a complete lockdown in 2020 due to Covid-19!), she/he will have to incorporate these features in the model as well. A less extreme example of this can be to include the feature of competition this shoe store faces or include the impact of opening of new stores on the sales of an existing store. Thus it is necessary to establish the width of uncertainty the forecaster desires to include.

It is said that 70%-80% percent of the effort while modeling goes into the data preparation and cleaning stage. From the discussion above, we can clearly see how consideration of various factors can help improve the forecasts in terms of accuracy, durability and consistency. No methodology fits all, one can only find out what works best for their problem by trying and improvising as they move along!