More about Net Promoter
As we discussed in Survey Pain Relief, Net Promoter, as promulgated by Fred Reichheld, is fraught with numerous difficulties on multiple levels. We discussed several at length in the book and so we’ll only enumerate them here. Thereafter, we will indicate and discuss several other difficulties what we did not stipulate in the book. As will be apparent, Net Promoter has many problems over and above ones we already indicated. Last, we will say a few words about what is right and worth keeping in the Net Promoter concept.
Net Promoter Flaws from Survey Pain Relief
1. The same score can be achieved in wildly different circumstances
2. Two concepts (“belief in superior value provided by the firm” and “feel good about the relationship”) are touted to be collected in a single measure – a violation of standard scientific measurement practice.
3. As adapted by several companies, the 0-11 system of measurement can also be 0-10, 1-3, and even the net can be discarded. At some point, one could argue that no real system is in place at all if each and every alteration “works”.
4. The claim that Net Promoter is the SINGLE metric needed to drive performance is contradicted by the additional measuring and drilling down Reichheld himself states would be necessary so as to ascertain what was awry and hence in need of positive change.
Conceptual Flaws in Net Promoter
Many writers have focused on and criticized (or praised) Net Promoter from various angles. Douglas Grisaffe published one such paper in the Journal of Satisfaction/Dissatisfaction and Complaining Behavior (volume 20, 2007) which focused on conceptual flaws, and we rely on his work in large part for the material we share here.
1. Recommendations alone are not enough. There is a compelling case to be made that recommendations, while powerful, are not necessarily the only or best means to guide a company. What about increasing current customer sales volume, cross selling, or looking to share-of-purchase compared to competitors? Will the recommendation piece be more powerful than marketing activities such as promotions, advertising, celebrity endorsements, etc. targeted toward the acquisition of new, potentially highly profitable customer segments? What effect, if any, does NPS have on customer lifetime or customer lifetime value? All these questions derive from more traditional theories of customer loyalty that have long been espoused and studied in both academia and management practice. It is interesting to note that Reichheld himself has addressed some of these more traditional ideas in his own writings.
2. Definitional concerns. One mark of good science is attention to the definitions of concepts as a foundation of all else that follows. The definition of loyalty Reichheld provides is at odds with the definition utilized by previous researchers such as Oliver (1999) and others. When a meaningfully different definition is proposed, it is the responsibility of the author to compare and contrast the proposed with the accepted and specify why the proposed is expected to be superior. This was not done. The definition Reichheld uses is as follows:
“Loyalty is the willingness of someone
—a customer, an employee, a friend—
to make an investment or personal sacrifice
in order to strengthen a relationship.”
Grisaffe provides a particularly apt example of someone willing to make investments and personal sacrifices to strengthen a relationship, but it looks very little like loyalty, as most of us would understand loyalty. Imagine a man who accumulates sexual conquests with the same intensity that Jay Leno collects classic cars. The man would be willing to buy flowers and expensive dinners, spend time to find out the likes and dislikes of the target, etc. in order to “seal the deal” and this would clearly qualify as personal sacrifice and making an investment. But once the goal is accomplished, the man moves on to another potential conquest and this moving on looks nothing like loyalty.
A bit closer to home for theorists is the intriguing possibility that one could be willing to recommend and therefore qualify as a promoter while simultaneously not intending to repurchase the product or service oneself. Many theorists agree that repurchase intentions do not equate to loyalty; nevertheless we have never heard that loyalty could, by definition, exist in the absence of repurchase intentions. Exactly what is to be made of this from a foundational, definitional point of view is not covered in Reichheld’s writings.
3. The Nature of NPS. Theorists not only start with well conceived, clear definitions of the variables and concepts of interest, but they also settle on the nature of the concept; that is, is it an outcome of loyalty, a measure of loyalty or a cause of loyalty? When various paragraphs are read from Reichheld’s works, it is not clear what his conceptualization is as it sometimes touted as a determinant, sometimes an indicator, and at others an outcome. The “nature” question is not trivial as appropriate managerial actions could only occur once we know what the variable is. If NPS is a determinant of loyalty, then proper managerial action may be to drive this score itself higher. In this case, one could propose more careful selection of participants so that those more likely to recommend are selected to participate. If it is a consequence of loyalty, then focus on loyalty itself may be necessary so as to drive the score. In this case, various loyalty programs may be implemented such that repeated use of our firm results in additional benefits to the customer, hence enhancing loyalty, which will then drive the NPS score. The nature of the conceptualization, then, is both scientifically and practically necessary as our examples demonstrate.
4. Correlation is not causation. Reichheld contends that NPS correlates with business performance. Correlation simply looks as whether two variables co-vary with each other and we typically refer to this co-variance as correlation. For example, one may wish to know whether a college entrance exam such as the SAT co-varies with performance in college as measured by earned grades. Alternatively, one may wish to ascertain whether a paper-and-pencil test, which purports to measure anxiety co-varies with subsequent performance on a business statistics exam. In the first case, the variables would be expected to be positively related in that SAT scores and college grades would likely “go together” in that high SATs and high grades would go together and the reverse with low scores. In second case, a negative relationship could be expected in that the scores will mirror each other such that when anxiety scores are higher scores on a test in business statistics would be lower and the reverse for lower anxiety scores paired with better outcomes on a business statistics test. Note that in neither case could we truly say that the pre-measures caused the post-measures—only that the scores co-varied.
Now where causation is present, co-variation is also present; that is, when “A” causes “B”, “A” and “B” also co-vary. But “A” and “B” can co-vary when causality is absent as when “A” and “B” share an underlying cause “C”. Looking again at the SAT and college grades correlation, the actual underlying causation of the correlation is likely to be a combination of intellect, work ethic, and educational level attained by the parents (among other variables which have been well-studied in education research). In the case of the anxiety test and business statistics performance, the underlying “C” variable is the condition of anxiety that tends to interfere with concentration on tasks which REQUIRE concentration such as passing exams in business statistics. (In fact, sufficient anxiety can cause “freezing” which is the inability to do anything at all!
All of this has been said to clarify that NPS may be correlated with some measures of business performance, but the relationship cannot be claimed to be causal. It is interesting to note that Reichheld himself made this distinction in previous writings, but abandoned it in The Ultimate Question.
5. Temporal Precedence and Causality. On a related issue, in order for causality to exist, temporal precedence must also exist. That is, for “X” to cause “Y”, “X” must occur before “Y” and must do so every time. Logically then, it is impossible to infer the Net Promoter Scores drove business success if the business success occurred, partially or in total, prior to computing the score. This is actually what happened and was published in the 2003 Harvard Business Review article. The situation was that the measure of corporate growth used covered the period from 1999 to 2002. The NPS measurement activity was not initiated until 2001. Therefore, something that happened in the future (the NPS measurement) is being used to predict what happened (at least partially) in the past. As the period of 1999 to 2001 had already occurred, it is clear that the NPS measure initiated in 2001 could not be causally related to the business outcomes which had already occurred.
So what is worth keeping? What has Reichheld meaningfully contributed to discussions of loyalty and business growth?
The very public success of the NPS has resulted in an unprecedented level of interest in customer loyalty and its relationship with business success. It has long been accepted that customer loyalty is an important ingredient in business success for most goods and services where repeat business is realistic possibility as it is in most consumer goods.
Moreover, Reichheld has brought to the forefront the importance of useable measurement to be used to compare against business goals as a key to success in managing and driving a business to greater success. Specifically he says:
“Companies won’t realize the fruits of loyalty
until usable measurement systems enable
firms to measure their performance against
clear loyalty goals – just as they now do in the
case of profitability and quality goals.” p.49
The focus on good measurement in customer satisfaction and loyalty is long past due.
The passion for his subject and the ready adoption by several major companies make his work singular in American business. However, NPS is not a panacea and no company has ever adopted it in a literal sense. Additional measures are needed to make NPS useful for driving a company to future success. NPS is NOT “the one number you need.” |
