Privacy Policy

Nine Priciples for Forecasting in Survey Analysis (With Scientific Bibliography)

While I was building a forecast model and I was looking for scientific confirmation regards some assumptions I discovered an interesting book, really suggested “Principles Of Forecasting: A Handbook for Researchers and Practitioners” written by J. Scott Armstrong, from Wharton School.

This handbook deals with all kinds of decision models.

It starts from the judgmental (Expert Opinions, Surveys) to econometrics model, multivariate analysis and neural network application for forecasting.

More then 600 hundred pages, rich and full of information and each chapter is written by a different professor from top universities.

Photo by Markus Spiske on Unsplash

This post is a summary of a chapter about Customer Intentions and behaviors and made me think.

Why?

Because I used a lot of surveys for my first startup, I often read reports based on surveys and interviews.

For example this year we made a survey with Mosiello (InnovAction Lab alumnus) in order to evaluate and improve the InnovAction Lab Annual BBQ.

Mosiello (un alumno di InnovAction Lab) per valutare e migliorare il BBQ Annuale di InnovAction Lab.

The chapter “Methods For Forecasting From Intentions Data” is written by Vicki G. Morwitz, from New York University.

First, we begin defining the word “Intention” in order to have a common language.

There are many definitions, but for the purpose of this post and for consistency with the book I will use the following one:

Intentions are a measure. Specifically, they represent a level of possibility for individuals to achieve plans, goals and future expectations. Often intentions are used to estimate what peoples are willing to do in the future.

Morwitz has developed nine principles to drive decisions based on information gathered by “customer intentions”.

Morwitz highlights that even using this nine principles, we must be careful making forecast based on intentional data.

Always.

Nine principles developed by the author are based on the following questions:

  1. How should be measured Intentions?”
  2. How should be used intentions to forecast behaviors?
  3. How should tune/adapt intentions when we have to forecast behavior?
  4. When declared intentions/preferences should be used to forecast/predict a behavior?
  5. Why should not be consistency between intentions and the actual behavior?
  1. How should be measured Intentions?

    • Using a probability scale, instead of other classification methods. (First Principle)
    • We must explain to the respondents to be extremely realistic on their effective expectation and personal characteristic. (the classic example here is the wish to have a Ferrari and the money to afford this expense) (Second Principle)
  2. How should be used intentions to forecast behaviors?

    • It should be avoided to use “raw” measured “Intentions”, they should be preprocessed before using to make a forecast. For example, if intentions are evaluated using a probability scale a quick and easy way to evaluate a percentage of potential buyers could be defined as the average probability to buy between respondents(Third Principle). In general, some researches showed the probability of buying durable goods is often underestimated compared to effective sales*. Bird and Ehrenberg studies showed an inverse trend for nondurable goods: intentions overstate purchases**
  3. How should be adapted/tuned intentions when we have to forecast a behavior?

    1. We can use past survey data and based on that results adapt/tune latest intentional data (Fourth Principle)
      1. Effective Behaviour(t)=Average Intentions Value (t-1) + Bias(t-1,t)
      2. Bias(t-1,t)=Effective Behaviour(t) –Intentions Average Value (t-1)
      3. Effective Behaviour (t+1)= Average Intentions Value(t) + Bias(t-1,t)
        • (t -1) time of the previous survey
        • (t) time of the latest survey that we will use to drive new strategy or forecast, is also the time when we measure the effective behavior of the measured intentions a time t-1
        • (t+1) represents the time when expressed intentions at the time (t) will be achieved
        • Bias, is the error, made at the time (t-1), it is a function of actual behavior at time t
    2. Through a segmentation and clustering the respondents before adapt/tune intentions (Fifth Principle)
      • Morwitz e Schmittlein analyzed in details the previous point, splitting respondents between who was intentioned to buy and who was not intentioned:

Authors from their studies evaluated that segmenting respondents through methods, where dependent variables (Criterion) and independent variables (Predictor) were identified, followed a lower forecast error compared with aggregated values.

In the specific case, they analyzed different households and their attitude to buy a car in one group and to buy a PC in another.

The techniques for clustering into intentioned and not intentioned buyers were highly cutting-edge (research was done in 1992) and they did not settle for only one method, they evaluated different methods, some well-known to all DataScience and Analytics enthusiast:

  1. Aggregate By income
  2. Using K-Means algorithm based on demographic data and product variables
  3. Discriminant Analysis where the purchase was predicted based on some demographic and product usage variables
  4. CART (Classification And Regression Trees), the prediction was based on demographic insight, product usage data, and other independent variables
    • Mentioning the paper inside the chapter: “The main empirical finding is that more accurate sales forecasts appear to be obtained by applying statistical segmentation methods that distinguish between dependent and independent variables (e.g., CART, discriminant analysis) than by applying simpler direct clustering  approaches( e.g., a priori segmentation or K-means clustering)“****
Independents Variables  (Predictors) are identified in the Morwitz e Schmittlein research
  1. Use the intentions measures to define boundary/limits of purchase probability (Seventh Principle).
    • When you have to evaluate the best and worst scenario, authors point out that is possible to use extreme values in measured intentions
  2. When intentions and preferences should be used to predict a behavior?

    1. According to the study by Armstrong, there are six conditions that determine when reported intentions should be predictive of behavior (Seventh Principle)***
      1. The predicted behavior is important
      2. Answers are made by the decision maker
      3. Respondent has a plan
      4. Respondent can clearly describe the plan (opposite to the respondent who states “I can’t tell you”, or describe the plan in a vague manner or the plan is not consistent with the attitude of the respondent)
      5. Respondent can achieve what he planned
      6. New data and insights are unlikely to change the plan over forecast time span ***
    2. Why could happen that intentions are not consistent measure compared with the actual behavior?
      • Measuring intentions modify behavior. Morwitz, Johnson, and Schmittlein, in one of their studies, selected two groups that should buy a car. In one group they measured the attitude to buy, in the other was not measured. The result was that in the group where buying intentions were measured more people actually bought a car. The explanation lies on the cognitive plan, when the respondent answer he triggers a “process” that Goleman in his book “Emotional Intelligence” (strongly recommended to all, you can find here) call “Metacognition” (Eighth Principle)
    3. Often people are affected by cognitive bias when they have to recall when last purchase was done(Ninth Principle) If this bias is in the answers then forecast and the prediction could be distorted. It’s important to identify it, in order to define an uncertainty level in the forecast. The ninth principle is the result of Kawani and Silk’s researches

I found these principles really helpful and interesting. In particular, the first three are quick to use, the last more complex and they must be looked more closely.

And you?

If you liked or you find it useful, share it through the social networks, with just one click can raise an opportunity.
If you think something needs to be fixed, you found any typo, write me!
Thanks for reading the article!

Andrea

*(Juster 1966 “Consumer buying intentions and purchase probability: An experiment in survey design”, McNeil 1974, “Federal programs to measure consumer purchase expectaions”)

**(Bird, Ehrenberg 1966 “Intentions-to-buy and claimed brand usage”)

***(Armstrong, Long-Range Forecasting , pag 83)

****(Vicki G. Morwitz and David Schmittlein,Nov., 1992, “Using Segmentation to Improve Sales Forecasts Based on Purchase Intent: Which “Intenders” Actually Buy?”)

*****(Silk Urban, “Pre-Test-Market Evaluation of New Packaged Goods: A Model and Measurement
Methodology”,Journal of Marketing Research, Vol. 15, No. 2. (May, 1978), pp. 171-191.)

 

 

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.