A New Approach to New Product Forecasting
Mike Gilliland
Product Marketing Manager
- New to the world products (entirely new types of products)
- New markets for existing products (such as expanding a regional brand nationwide, or globally)
- Refinements of existing products (such as "new and improved" versions, or packaging changes).
There are many NPF approaches available to try. Some of the common ones include:
- Executive Opinion – top management provides the forecast
- Sales Force Rollup – a bottoms-up poll of the sales people force
- Delphi Method – a structured formal process for anonymously gathering forecasts and building a consensus
- Prediction Market – anonymous wagering used to gather group opinion
- Analogy – expecting a new product to behave like similar products from the past.
All of these methods use judgment to some extent, and there are good reasons why. Judgment compensates for the lack of historical information – we are dealing with new products with no historical sales. Judgment also compensates for lack of future information – it may be too difficult or costly to conduct market research tests, or to quantify such things as the direction of fads and fashion trends.
While use of judgment is necessary in new product forecasting, it has its disadvantages as well. Judgment is frequently biased with over-optimism, or allowing recent events to have unwarranted impact. Judgment is also clouded by personal or political agendas, where the forecast is used to represent what the person wants or needs to have happen, rather than what they honestly believe will happen.
As an example, if a sales rep knows his forecast is going to be used to set the sales quota, there is a natural tendency to forecast low – to set a low quota that is easier to beat. If a product manager wants to introduce a new product, there is a natural tendency to forecast high – at least high enough to meet any hurdles for getting the new product approved for development! (Have you ever heard of anyone forecasting their new product idea is going to fail in the marketplace?)
The use of analogies is a common NPF practice. We see this in the real estate market, where an agent will prepare a list of "comps" – similar houses in the area that are on the market or have recently sold – and use this to suggest a selling price.
SAS has a new patent-pending approach to NPF that combines the use of analogies with structured judgment. This "structured analogy" approach has these features:
- Guided statistical analysis that incorporates human judgment
- Attempts to remove judgmental bias by providing a historical context for each decision
- Attempts to validate and test the decisions
- Choice of analogy is driven by a statistical process.
The structured analogy approach requires two types of data: product attributes (for prior and new products), and historical sales (for prior products). Product attributes can include many things such as:
- Product type (toy, music, clothing, shirts, etc.)
- Season of introduction (summer-item, winter-item, etc.)
- Financial (own-price, competitor-price, etc.)
- Target market demographic (gender, age, income, ethnicity, etc.)
- Physical characteristics (style, color, size, etc.)
- ... and many others.
Historical information on past new product introductions is also needed. Here are scaled thumbnails of the first eight weeks of sales for 100 new DVD releases:
The structured analogy process for new product forecasting has six main steps:
- Query Step – find a set of candidate products that have similar attributes to the new product.
- Filter Step – manually remove inappropriate or outlier products from the set of candidate products.
- Cluster Step – cluster the candidate products according to their sales pattern, and manually select the most appropriate cluster to serve as the surrogate products.
- Model Step – select the most appropriate statistical model for the cluster of surrogate products, and extract the statistical model features.
- Forecast Step – use the extracted statistical model features to forecast the new product.
- Override Step – make manual adjustments to the statistical model's forecast.
Let's work through an example of forecasting sales of a new DVD movie release.
The Query Step begins with selecting a publicly available dataset on historical DVD sales, and then specifying the attributes of prior DVDs that match the new DVD. In this case, the new DVD is an R-rated Horror movie, so we decide to specify two attributes: "Horror" for the Genre, and "R" for the MPAA Rating.
Note that we are using judgment to determine which attributes are most relevant – the system is not going to tell us this. However, the system does automate all the work of extracting the R-rated Horror movies from the dataset of all DVDs, and these form our pool of candidate products.
The output from the Query step is a profile overlaying the initial sales of all the candidate products, along with a list of all the candidates.
Judgment again comes into play in the Filter Step, as we decide that "Dawn of the Dead" is an outlier, and uncheck it to remove it from the candidate pool.
The Query Step let us explore candidate DVDs with similar attributes, and the Filter Step allowed us to make a judgment on inappropriate candidates to be filtered out. After the filtering is applied, the Cluster Step clusters the remaining candidates according to similarity of their sales pattern. Judgment again comes into play, as the user selects a surrogate cluster based on the anticipated sales pattern for the new product. In this case the R-rated horror movies fell into three clusters, and the first was chosen. We are now ready for the Model step.
The Model Step generates a statistical model fitting the surrogate cluster. The user has access to a variety of models to utilize, including diffusion, mixed, smoothing, Bayesian, and other types of models. After the model is selected, the Forecast Step generates a forecast for the new product, which is shown below in blue overlaid on the surrogate product histories.
Judgment has been used throughout the structured analogy process, and is used once more in the Override Step, where manual adjustments can be made to the statistical model forecast.
After any manual overrides are made, the new DVD forecasts can be exported to downstream planning systems.
The structured analogy approach can be useful in many, but certainly not all new product forecasting situations. It attempts to improve on human judgment alone by automating the historical data handling, and incorporating statistical analysis. The software makes it possible to quickly extract candidate products based on the user-specified attribute criteria. It aligns and scales and clusters the historical patterns automatically, providing an easy to understand visualization of past new product behavior. This visualization helps the forecaster realize the risks and uncertainties and variability in new product behavior, so that the organization can make the appropriate decisions based on these uncertainties.
Judgment is always going to be a big part of new product forecasting. A computer will never be able to tell us whether Lime Green or Day-Glo Orange is going to be the hot fashion color next fall. But judgment needs assistance to keep it on track and as objective as possible. The role of structured analogy software is to do the grunt work, making the NPF process as automated, efficient, and objective as possible.
For more information on this topic, see the recorded webcast "Alternative Approaches for New, Low Volume, or Highly Seasonal Items" that is available for free replay at www.bettermanagement.com. New product forecasting is also a topic at the Institute of Business Forecasting's Consumer Products and Retail Forecasting Forum in Las Vegas (part of IBF's Best Practices conference running April 29 - May 1). Forum panelists include SAS, Timberland, Carhartt, Wells Dairy, Tredegar Film Products, and Bayer Cropscience. See details on the Forum and the full IBF conference.
