12 Forecast Errors
Vikas Singla
12.1 OBJECTIVES
This chapter would help students to understand:
- Two approaches to forecasting: point vs range forecast
- Concept and methods of Forecast Errors
- Mean Absolute Deviation (MAD) method
- Mean Square of Error (MSE) method, and
- Mean Absolute Percentage Error (MAPE) method
- A modern approach to forecasting: Accurate response technique
12.2 INTRODUCTION
Cut-throat competition in all sectors of economy, shorter product life cycles, increasing need to develop new products faster, ever need to increase variety of products is making it more difficult for manufacturers and retailers to predict which of their goods will sell and which would not. Such phenomenon also poses problems for retailers and distributors to evaluate which and how many products to order and sell. This result in occurrence of forecast errors i.e. there is increase number of happenings of substantial difference between actual and forecasted demand. This difference has been found to have significant impact in increasing costs. This difference can result in two scenarios: firstly, manufacturer has produced or retailer has stored more products than demand resulting in unwanted goods. These goods might become obsolete in near future so manager intends to dispose them off at low prices or even sold at a loss. Secondly, inaccurate forecast might result in loss of sales as goods are produced in lower numbers than demand. Such unpredictability of demand is quite apparent in fashion and apparel industries, food sector, various services etc.
Traditionally, inaccuracy in forecasts has been addressed by companies by applying various systems such as Just-in-time (JIT), inventory systems and the like. These systems were found wanting in case of increase in volatility in demand and uncertainty from suppliers. For instance, a manufacturer can rapidly change its production schedule with change in customer demand but would final to change the supply chain if it has already been filled with old raw material. In industries which cater to seasonal demand like hotels etc. sales are concentrated in few months during which manufacturer require large capacity whereas in other months this capacity can remain unutilized. Thus, accurate forecasting becomes highly important to smoothen the capacity requirements over a long term period.
12.3 TWO APPROACHES to FORECASTING: POINT vs. RANGE FORECASTING
Every business decision big or small involves prediction about future. How many people to hire, how much capacity to increase or decrease, when to launch new product are some of the decisions that are based on accurate prediction of demand. Thus forecasting accurately becomes an important and necessary aspect of decision making for every organization.
One of the most common approaches towards forecasting relies on point prediction. This approach tends to rely on predicting precisely the future. For instance, this is like asking a student how many marks he/she will score in upcoming statistics paper. It is very difficult for the student to tell the exact marks that he/she will obtain for an event which will happen in future. But most companies tend to follow this approach. They insist on asking exact amount of increase in sales next year or precise amount of profits that a subsidiary would generate. Another simile analogy is the sales target that a manager imposes on his/her subordinate. The subordinate has to achieve an exact amount of sales in limited time period. It becomes quite difficult for subordinate to achieve the same precise amount. A better approach is to forecast within a particular range. For instance if the student says that he/she is pretty sure of getting marks between six and eight out of ten in upcoming exam then it would be more achievable and reliable. This type of forecast is not accurate as it leaves some margin of error as candidate can sore any marks within that range. But it tends to provide more probable outcomes.
The Point Forecast
This approach of predicting exact value or judging the best guess has following shortcomings:
- Point forecast approach is helpful in predicting events which are certain to happen in near future. For instance predicting tomorrows’ weather would be easy to predict based on today’s’ weather conditions. But it would be difficult to pinpoint exactly what would be the temperature tomorrow morning. Thus, this approach should be applied to forecast for very short term periods and for those events whose outcomes are know with certainty.
- This approach ignores the fact that an event which is being forecasted is not a predetermined entity. The circumstances under which an event has occurred in the past might change in near future. For instance sales of a particular type of style and colour would change in near future with change in economy, fashion trends, consumer tastes etc. Thus instead of having precise outcomes it is better to have probability distribution for a series of events. Each probability value for each event would indicate the likelihood of occurrence of an outcome.
- This approach ignores inherent characteristic of uncertainty in future events. If possibility of an outcome is indicated by an exact value then the prediction is done with full confidence and no uncertainty is left. But future events would always some element of uncertainty. Thus this approach would consider circumstances of both volatile and stable conditions as same which might prove somewhat accurate for stable market conditions but not for unstable conditions. Awareness of uncertainty is necessary for managers to prepare their businesses.
The Range Forecast
Range forecast also know as confidence interval is another approach of forecasting where a range of probable outcomes lying between the best and worst case scenario is given. Telling your superior that you will reach the office exactly at nine in the morning is difficult to achieve than saying that you will reach office between 8:45 and 9:00 am.
The important consideration in estimating range forecast is the size of the interval. The decided size of the interval would indicate a trade-off between accuracy and confidence. If the goal is only of being confident then predicting that sales would be between zero and infinite would be correct. This range would certainly include true value but it would not be at all helpful for the managers as it does not provide any relevant information. On the other hand an accurate value of sales forecast say number of units that would be sold next year is 1000 would provide precise information but might not be achieved with full confidence. To counter the negatives of both cases researchers tend to follow confidence interval approach with the size of interval to be as precise as possible. Sales would increase by 10-12% is more informative and accurate than the remark that sales would increase by 5-20%. Hence, a manger while forecasting must make a trade off between being informative by narrowing the range included in the interval and being accurate by increasing the likelihood that the confidence interval would include true value by making it wider. This trade-off is often difficult if not impossible to achieve.
12.4 QUANTITATIVE METHODS OF CALCULATING FORECAST ERRORS
Forecast error is the difference between forecasted data and actual data. Higher the forecast error less accurate would be the forecasting technique used. The forecast error values can be positive or negative. Positive values would mean that actual demand was more than forecasted value which might result in loss of customer as demand for all customers is not satisfied. Negative forecast error values would indicate that actual demand was lower than forecasted value which would result in increase of inventory. Thus, accuracy of forecasting technique becomes paramount as higher forecast error could have harmful effect on a company’ operations.
12.3.1 Mean Absolute Deviation (MAD)
To obtain MAD of given data following steps should be followed:
Step 1: Calculate forecast errors i.e. find difference between forecast and actual data.
Step 2: Take an absolute of forecast error i.e. consider positive of even those forecast error values which are negative.
Step 3: Find the average of these absolute values.
12.3.2 Mean Square of Errors (MSE)
To obtain MSE of given data following steps should be followed:
Step 1: Calculate forecast errors i.e. find difference between forecast and actual data.
Step 2: Take square of each forecast error value.
Step 3: Find the average of these squared values.
12.3.3 Mean Absolute Percent Error (MAPE)
To obtain MAPE of given data following steps should be followed:
Step 1: Calculate forecast errors i.e. find difference between forecast and actual data.
Step 2: Take an absolute of forecast error i.e. consider positive of even those forecast error values which are negative.
Step 3: Divide each absolute value by actual demand value and multiply it by 100 to get data in percentage terms.
Step 4: Find average of these calculated percentage values.
The computation of methods of forecast error has been illustrated in following example.
Example 12.3.1: Data in table 12.3.1 pertains to actual and forecast demand for 10 time periods n the past. Compute the measures of forecast accuracy.
| Table 12.3.1 | ||||||||||
| Time period | 1. | 2. | 3. | 4. | 5. | 6. | 7. | 8. | 9. | 10. |
| Actual Demand | 120 | 114 | 130 | 124 | 97 | 95 | 100 | 110 | 109 | 123 |
| Forecast demand | 109 | 118 | 132 | 110 | 110 | 105 | 98 | 95 | 104 | 110 |
Solution:
| Table 12.3.2 | ||||||
| Time Period | Demand (D) | Forecast (F) | D-F | Absolute value of (D-F) | (D-F)*(D-F) | %age = (D-F/D)*100 |
| 1. | 120 | 109 | 11 | 11 | 121 | 9.17 |
| 2. | 114 | 118 | -4 | 4 | 16 | 3.51 |
| 3. | 130 | 132 | -2 | 2 | 4 | 1.54 |
| 4. | 124 | 110 | 14 | 14 | 196 | 11.29 |
| 5. | 97 | 110 | -13 | 13 | 169 | 13.40 |
| 6. | 95 | 105 | -10 | 10 | 100 | 10.53 |
| 7. | 100 | 98 | 2 | 2 | 4 | 2.00 |
| 8. | 110 | 95 | 15 | 15 | 225 | 13.64 |
| 9. | 109 | 104 | 5 | 5 | 25 | 4.59 |
| 10. | 123 | 110 | 13 | 13 | 169 | 10.57 |
| Total | 89 | 1029 | 80.22 | |||
Mean Absolute Deviation (MAD) = 89/10
= 8.9
Mean Square Error (MSE) = 1029/10
= 102.9
Mean Absolute Percent Error (MAPE) = 80.22/10
= 8.022
MSE value would always be greater as it compounds the error by square value. MAPE is a useful method if objective is to understand error in relative terms. MAPE can be used to compare forecasting methods in relative sense and then inference can be drawn. For instance, in this example MAPE is inferred as that there is an error of approximately 8 units out of 100. So, when this is compared with some other forecasting method then both values of MAPE would be compared at same standard value of 100. This provides much clearer picture than other methods. MAD is the most commonly method used to infer forecasting accuracy of a forecasting technique. This method takes into account contribution of each forecast error value and presents clearly any significant change in data over a particular time period.
12.4 APPLICATION OF FORECAST ERROR METHODS
Previous sections have discussed three different methods of finding errors in forecasting. This section would focus on discussing application of these methods by illustrating two examples. First example discusses which exponential smoothing coefficient would give more accurate forecast results by finding errors of two methods. Second example discusses which forecasting method is better to predict forecast results by using a particular forecast error method.
Example 12.4.1: Estimate demand for following data by using exponential smoothing method. Use alpha as 0.20 and 0.50 and suggest which alpha would provide more forecast accuracy.
| Time period | 1. | 2. | 3. | 4. | 5. | 6. | 7. | 8. | 9. | 10. | 11. | 12. |
| Demand | 65 | 80 | 90 | 100 | 110 | 90 | 75 | 55 | 50 | 60 | 50 | 45 |
Solution:
Time period |
Demand | Forecast | Forecast error (D-F) | Absolute value of Forecast error | Forecast | Forecast error (D-F) | Absolute value of Forecast error |
| α=0.20 | α=0.50 | ||||||
| 1. | 65 | ||||||
| 2. | 80 | 65 | 15 | 15 | 65 | 15.00 | 15 |
| 3. | 90 | 68 | 22 | 22 | 72.5 | 17.50 | 17.5 |
| 4. | 100 | 72.4 | 27.6 | 27.6 | 81.25 | 18.75 | 18.75 |
| 5. | 110 | 77.92 | 32.08 | 32.08 | 90.62 | 19.38 | 19.37 |
| 6. | 90 | 84.33 | 5.66 | 5.66 | 100.31 | -10.31 | 10.31 |
| 7. | 75 | 85.46 | -10.46 | 10.46 | 95.15 | -20.16 | 20.15 |
| 8. | 55 | 83.37 | -28.37 | 28.37 | 85.07 | -30.08 | 30.07 |
| 9. | 50 | 77.70 | -27.70 | 27.70 | 70.03 | -20.04 | 20.03 |
| 10. | 60 | 72.16 | -12.16 | 12.16 | 60.02 | -0.02 | 0.019 |
| 11. | 50 | 69.72 | -19.72 | 19.72 | 60.01 | -10.01 | 10.01 |
| 12. | 45 | 65.78 | -20.78 | 20.78 | 55.00 | -10.00 | 10.01 |
| Total | 221.53 | 171.29 |
MAD with α=0.20
MAD with α=0.50
= 221.53/12
= 20.13
= 171.29
= 15.56
From the above results it can be interpreted that as MAD is less when smoothing coefficient of 0.50 is used. Thus, smoothing coefficient of 50% should be used for forecasting purpose in this case as compared to 20%.
Example 12.4.2: Following data gives actual demand for an item for a nine month period. Test two forecasting methods to see which method was better over this period.
(a) Forecast April through September using a three month moving average
(b) Use exponential smoothing with an alpha of 0.3 to estimate April through September
(c) Use MAD to decide which method produced better forecast over six month period.
| Month | Actual | (a) 3 month moving
average |
Error | (b) Exponential
Smoothing |
Error |
| January | 110 | ||||
| February | 130 | ||||
| March | 150 | ||||
| April | 170 | 130 | 40 | 150 | -20 |
| May | 160 | 150 | 10 | 156 | -6 |
| June | 180 | 160 | 20 | 157.2 | 2.8 |
| July | 140 | 170 | -30 | 164.04 | 5.96 |
| August | 130 | 160 | -30 | 156.828 | 3.172 |
| September | 140 | 150 | -10 | 148.7796 | 1.2204 |
| MAD | 23.33 | 6.52 |
| ( c ) | Forecast error by using MAD for 3 month moving average | = 23.33 |
| Forecast error by using MAD for exponential smoothing method | = 6.52 |
As forecast error calculated by using MAD for exponential smoothing method of forecasting is less than forecasting error by using 3 month moving average method, so it can be inferred that exponential method should be used for forecasting purposes.
12.5 ACCURATE RESPONSE: A MODERN APPROACH TO REDUCE FORECASTING ERRORS
12.5.1 Demand Uncertainty
Most companies fail to incorporate uncertainty in demand of various products that they are manufacturing in to their production schedule resulting in inaccurate forecasts. They base production planning on forecasts which are made far in advance of selling season. For instance toy manufacturers forecast demand for toys far in advance of selling season which is during festive season of a month or so. This is done to provide them ample time for production during lean season. But when time comes for selling i.e. festive season there could be a scenario that demand changes and products become obsolete. This would cause huge stocks of unwanted goods. The failure to incorporate demand uncertainty happens for following two reasons:
- Companies fail to factor multiple demand scenarios into initial forecast. Demand can vary because of various reasons and if all these reasons are not accounted for then it would result in inaccurate forecast.
- Secondly, frequent change in consumer buying patterns. This would force companies to introduce new or variants to existing products. Flexible manufacturing systems and processes to produce small at low prices have facilitated in fulfilling varying demand of customers. But this has also resulted in certain side effects. For one, such frequent introductions reduce the average lifetime of products. Some of them die at introduction stage and some of them reached decline stage quite quickly. For instance, mobile phones have very small life cycles because of competition causing companies to launch new models very frequently. Similar scenario can be seen in case of automobiles. Also, as products proliferate companies indulge in forecasting for entire product line rather than forecasting for each variety of products. Such aggregate demand is easier to predict. But, forecast distribution among products which would give more accurate results is difficult. For instance, a soap manufacturer if is producing ten variants of soaps then the manager can forecast for all ten soaps in totality or for each soap differently. Aggregate approach would provide a sales forecast for all soaps but it would fail to indicate which soaps are less in demand and which are high in demand. Such approach would cause company to manufacture all soaps in equal quantity. Whereas if forecasting is done individually then it would be much clearer that which soaps should be manufacture in lesser numbers and which in higher numbers.
Following cases illustrate such problems:
Case 1: Retailers are exercising more power with suppliers in ordering of various products. A retailer selling toys would store toys only at the time of selling and would not store much before selling period resulting in shorter lead times. Also, the retailer formulated a policy to purchase toys depending on customer reaction. Earlier supplier expected to supply consignments of toys to the retailers’’ distribution centre and get paid. Now payment is deferred till actual sale has been made. Remaining toys would be sent back to manufacturer. This has made manufacturer to apply better forecasting methods so as to reduce building of unwanted goods.
Case 2: Usage of tools such as customized screw drivers, drills etc. as gift items were found to be a new phenomenon by a retail manger. It was also observed that customers want these items to be in attractive packages and in small sizes. Manufacturer had facilities to produce such tools in bigger sizes and less in number. So to fulfil new orders pertaining to change in customer tastes the manager has to change its production planning. Proper forecasting would have mitigated such problems in production.
Case 3: A car company like General Motors, Ford or Maruti Suzuki is involved in manufacturing variety of cars from SUVs to small cars. Demand for small cars was found to be much more than bigger sedans or SUVs. In case of Marutis’ small car which was manufactured in both petrol and diesel variants it became imperative for company to forecast for each variant of small car and for each segment. Aggregate forecasting approach would result in forecasting for all cars irrespective of segment and variant. This would cause faulty forecast without indicating demand for each type of car.
12.5.2 Accurate Response: The Technique
Accurate response has been suggested as another approach by manufacturers and retailers alike in reducing forecast errors. The technique was found to be quite appropriate in redesigning planning processes and minimizing inaccuracy in forecasts. It helps in:
- Figuring out what forecasters can and cannot predict well
- Making the supply chain more flexible and responsive so that managers can postpone their decision to buy raw material and produce for unpredictable items until they obtain some market signals regarding their demand.
This approach incorporates two basic elements which were absent in other traditional methods of forecasting:
- It considers missed sales opportunities. Forecasting errors result in either too low or too high inventory.
The technique of accurate response measures the cost of each unit in case of low and high inventory. These costs are then incorporated into the planning process. Many companies misses this aspect of sales lost and cost incurred on each lost sale.
- The technique distinguishes demand for the products which are predictable and which are unpredictable.
The items which have predictable demand can be produced much before sale season and then those free resources can be used to produce items with unpredictable demand which becomes known close to sale season.
Such knowledge helps companies to reconstruct and reconfigure their supply chains in terms of selection of vendors, transportation and location decisions and number of warehouses. It also encourages companies to redesign the products so as to meet changing needs with least disruption to production facilities. These two aspects of flexible manufacturing and shorter cycle times evaluated by accurate response technique are incorporated in planning processes which ultimately helps in accuracy of forecasting. Companies which make or sell products with stable demand and long cycle times can use traditional methods of forecasting which would prove to be quite accurate. But accurate response technique is quite useful to reduce forecast errors for companies which make or sell products with short life times and volatile demand.
12.5.3 Accurate Response Technique: A Case Study
Demand for seasonal products especially winter sports apparel is heavily dependent on number of factors such as weather, fashion trends and economy. Also demand for such products is heavy only for small duration i.e. during winter season which is for 2-3 months. A company (lets’ call it Winter sportswear) is involved in manufacturing such sports apparel. This case illustrates success of accurate response technique in reducing forecast accuracy for such seasonal products. The company had a 50 year long history and during 80s and 90s company enjoyed stable and increasing demand with nor or very less pressure on manufacturing facilities. But competition and changing needs of customer changed all that.
Challenges:
- With increase in sales of its products Winter sportswear faced manufacturing constraints as to fulfil increased orders it has to use or buy more resources putting huge pressure on existing resources. The law of economies of scale had reached its saturation point. During peak season because of saturation in manufacturing facilities, company was not able to take further orders. Thus to fulfil such orders they started pre-booking and focussed on producing much before the season.
- Demand for variety of products in small batches caused company to develop a very complex supply chain.
For example different zippers were required for different jackets making sourcing of such zippers from various vendors a complex task. The pressure to fulfil increased variety led to increased lead times.
- The company also launched a new product line catering to the needs of children. It was a successful venture and booking for such products was much earlier than apparels for adults.
Response to challenges:
- The company introduced new computerized systems which helped them to calculate raw material requirement for each type of product and aided in processing of orders.
- By bifurcating items into predictable demand and unpredictable demand items company started to stock raw material for predictable items much before the start of their production. This was done to shorten the time of procurement of raw materials. If it would have done at a later stage then such procurement takes much longer lead times.
- The company adopted more interactive systems with their retailers in order to judge and respond to changing customer needs. This helped company to persuade retailers to book their orders as early as possible at a discount. Such a scenario would help company to use its resources during lean season for producing predictable items and thereby freeing resources for unpredictable items which would be manufactured close to demand season.
- Lastly and most importantly company involved managers in forecasting aggregate demand and demand for individual items. It was found that company had adopted aggregate demand forecasting approach which blinded the items with low demand. For instance it was judged that which style and color is in huge demand which are in mild demand and which have low demand. It was found that some styles and colors were in huge demand whereas others had reached their decline stage. Also, after segregating product demand wise accurate response technique also helped company to segregate items into predictable and unpredictable items. This helped the company to push production of predictable items to much earlier in the season and for unpredictable items to much later in the season. Thus same resources could be used to fulfil increased quantity of orders streamlining the supply chain and manufacturing capacity.
Results:
Winter sportswear by applying two pronged strategy of refinements in supply chain and modifications in design by using accurate response technique was able to reduce its cost significantly. Supply chain changes focussed on keeping raw materials and production capacity as undifferentiated as possible i.e. same resources could be used to produce variety of products. Also focus on modification in designs taking into consideration customer tastes was helpful in unlocking supply chain. For example instead of using different colored zippers for different apparels company started to using same colored zipper in all its apparel. The move was accepted by the customers which increased sales and reduce costs in procuring different zippers from different vendors.
12.6 SUMMARY
Two approaches to forecasting namely point forecast and range forecast has been discussed. Point forecast provides confidence to forecast but tends to be inaccurate most of the time as an exact value prediction is near impossible to achieve with change in circumstances. On the other hand range forecast provides accuracy to forecast as true value has high likelihood of falling in the given range but lacks confidence in prediction. Also challenge to this approach is selection of size of interval. Different methods of evaluating forecast accuracy have been discussed. Mean Absolute Deviation (MAD), Mean Square of Error (MSE) and Mean Absolute Percentage Error (MAPE) has been discussed with illustrations. These methods can be applied to find which of discussed methods of forecasting help in predicting more accurately. Also a modern approach to forecasting namely accurate response technique has been discussed to emphasize the importance of flexibility in supply chain and modification in design of products. A case study illustrated the application of this approach.
12.7 GLOSSARY
- Forecast Error: is the difference between forecast and actual demand indicating accuracy of forecasting technique.
- Mean Absolute Deviation (MAD): is the average forecast error using absolute values of the error of each past forecast.
12.8 REFERENCES/ SUGGESTED READINGS
- Chase, B.R., Shankar, R., Jacobs, F.R. and Aquilano, N.J., Operations & Supply Chain Management, 12th Edition, McGraw Hill.
- Stevenson, W.J., Operations Management, 9th Edition, Tata McGraw Hill.
- Lee J. Krajewski, Operations Management, Prentice-Hall of India, New Delhi, 8th Edition
12.9 SHORT ANSWER QUESTIONS
1. Accuracy of different methods of forecasting can be evaluated by applying forecast error techniques.
(a) True (b) False
Answer: a
2. Which forecast accuracy method would give the highest value?
a. MAD b. MSE c. MAPE
Answer: b
3. Which of the following methods could be used to calculate forecast error in relative terms?
a. MAD b. MSE c. MAPE
Answer: c
12.10 MODEL QUESTIONS
1. Contrast the use of MAD and MSE in evaluating forecasts.
2. For the following data use MAD, MSE and MAPE technique to calculate forecast error and interpret the importance of each.
| Period | Actual | Forecast |
| 1 | 700 | 660 |
| 2 | 760 | 840 |
| 3 | 780 | 750 |
| 4 | 790 | 835 |
| 5 | 850 | 910 |
| 6 | 950 | 890 |