At the 2014 CASE District IV Annual Conference, I presented on alumni opinions and feedback with Graham Stewart, Vice Chancellor for Alumni Relations, at Vanderbilt University. At that meeting, Graham spoke about his use of the Net Promoter Score (NPS), a process taken from top business consultants that centers on asking stakeholders about their willingness to promote the institution to others. Darrow Zeidenstein, Vice President for Development and Alumni Relations at Rice University, asked several interesting questions; Are universities the same as businesses? Should universities focus so heavily on a question about willingness to promote as opposed to a one about satisfaction, decision to attend, current opinion, or loyalty?
To answer Darrow’s question, we looked at our data set of over 500,000 alumni opinions from a wide range of universities and colleges. In short, our answer is that the promoter question is not the best question to measure alumni willingness to engage with their alma mater. There is a question that works better, one that consistently ties to engagement and giving. The following discussion is a brief review of why we came to this conclusion, what we love about the NPS, and what we recommend instead of the promoter question.
Before getting into that, we need to better understand the intended use of the NPS and its application for higher education. For a business, the NPS comes out of the transactional nature of the relationship a company has with its customers. For higher education and especially alumni relations, the model is a bit more complicated. First, alumni relations professionals want to increase the attendance at their events. This has a transactional element and in that way, is similar to the problem a business faces trying to increase sales. On another level, universities are trying to increase the probability that alumni will include their alma mater in their lifetime giving decisions.
We know that some percent of students graduating this year will, at some time in their lives, find themselves in a position to make a transformative gift to their alma mater. The problem is that we have no idea which alumnus/a will be the one with that circumstance. Given this problem, alumni relations professionals need to focus on increasing the probability that as many alumni as possible feel engaged with their alma matter while that alumnus/a is in his or her twenties or early thirties. This is the sweet spot advertisers talk about when they worry about ratings. It is the time in our lives that we set priorities. Universities that successfully engage alumni during this period have the best chance of being included in the thinking of those alumni who find themselves in a position to make a major gift later in life. Also, the university is better positioned for planned giving from other alumni.
The NPS is an exciting process that can invigorate an organization, but it is only as good as there is faith that the question asked is pointed in the right direction. Success with the NPS process occurs in organizations that are clear about their goals and believe that the question used in the process is steering them towards those goals.
The best place to start a discussion of the NPS is at the source, Fred Reichheld’s book titled The Ultimate Question and its follow-up book, by Richard Owen and Laura Brooks, titled Answering the Ultimate Question. These books came out of extensive research and applied experience based analysis compiled by Bain & Company: Global Management Consulting Firm. The key idea Bain & Company wanted to shed light on was how U. S. companies were addicted to what Reichheld referred to as “bad profits.” We are all familiar with these “bad profits.” They show up when you are charged an additional fee for bags or a blanket by your airline or when your credit card company charges outrageous fees for a late payment. Reichheld argues that corporations are addicted to bad profits at the expense of customer satisfaction and they need a way to measure corporate success through a lens that also accounts for customer loyalty.
The strength of the NPS comes from two places. The first is about focusing the organization around one question as a way to measure progress. For Enterprise Rental Car and Southwest Airlines, this ability to focus the organization on a single measurement made all the difference. When research is driving behavior, real change is not only possible, but also probable.
The trick, or second critical part of success, is to find a question that resonates with the organizational team and its management. Success depends on the ability to find a question that has been vetted and is clearly positioned to move the entire organization in the desired direction. The power of the NPS is its ability to bring focus out of apparent disorder, but this is only possible if everyone agrees with the focusing agent: the question. Without clarity and purpose behind the question, the process will probably go the way of most other business fads.
To find the right question or questions, Reichheld reviewed Bain’s research and teamed with a survey company to gather more data about customer opinion and its impact on their behavior. Through that process, he found that a customer’s willingness to promote the organization had the highest impact on their continued purchases and commitment to the company. This resulted in what we now refer to as the “NPS” question. Reichheld is clear, however, that this may not be the right question for all companies. For example Enterprise Rental Car uses the following question: “How satisfied are you with your most recent rental experience?” Further, Reihheld uses a 10-point scale and Enterprise uses a 7-point scale. The point is that each company or industry needs to find the question that is right for it.
So, what is the right question for higher education? The right question should be the one with the highest correlation to something that matters most to the advancement of the university. Clearly, for a business focused on increasing sales and customer loyalty, “willingness to promote” makes sense, but is it the right question for a university? I can see how it might be, if the biggest need for the university is to increase enrollment or event attendance. If alumni are promoting the university to others, they are, in all likelihood, increasing awareness and potential enrollment or attendance. If, however, the university is focused on philanthropy and advancement, there might be a better question. To find the best question, we reasoned that it should be one that is most closely related to an alumnus/a’s intent to financially support the university.
Another goal of advancement and more directly, alumni relations, is attendance at events. Event attendance increases the probability that the alumnus/a attending that event will give, and more importantly, that he or she will consider the university over time in their long-term philanthropic activities. We know if an alumnus/a attends an event, there is a greater likelihood that same person will give to the university, and if they attend more events, there is a greater likelihood they will give more.
Since 2001, in almost all the Alumni Attitude Study projects PEG has completed (330 surveys, with more than 230 different universities and/or colleges), we ask questions about:
- Satisfaction Index: based on how respondents answer questions about willingness to promote, overall opinion of the university, experience as an alumnus/a, and decision to attend the university;
- How well the degree prepared alumni for various aspects of their life;
- What is the level of impact on the alumnus/a opinion of their alma mater among elements like value of degree, athletics, and/or accomplishments of alumni or students or faculty;
- What matters most about alumni programming;
- Where are the best places on campus to engage future alumni (students);
- What are the barriers to alumni participation;
- What matters most to alumni about communications and where the partake;
- What do alumni most want to know is better today than when they attended their alma mater; and
- A separate question that measures intent to financially participate.
With over 500,000 responses from alumni across the country, including 48% of the US News and World Report Top 100 Best National Universities, we are in a unique position to determine what question is best for higher education. As with the Bain Consulting Group, we know that once you have the data, the analysis is pretty straightforward. In the case of higher education, we looked for questions with the highest correlation to how an alumnus/a answered a question about their intent to financially participate with the university. We also looked at correlations to the PEG Satisfaction Index and to the individual elements of that index.
As it turns out, loyalty for the university is the overwhelming winner. This is true overall and across all of the different eras of graduation. Some of our clients have used “relationship with the university” instead of loyalty. Our findings show that relationship is a reasonable proxy for loyalty. The correlations to loyalty are higher, but both loyalty and relationship to the university have a higher correlation than willingness to promote.
The question with the next highest correlation is about the alumnus/a overall current opinion of the university. After that, we saw the next highest correlations with willingness to promote, success of athletics, and decision to attend. From this research, we recommend that universities transition from a NPS process to a Net Loyalty Score (NLS).
In my mind, these findings make a lot of sense. Much like Darrow, I never understood why an alumnus/a’s willingness to promote the university to others was a good measure of their engagement, especially in a broader sense than event attendance. That brings us back to the NPS or for universities, the NLS. I completely agree with the Bain team in their focus on this as a process. They argue that The NPS/NLS is a way of doing business, as well as a metric. That being said, it is important to use the right metric.
From what I can see, most universities implementing a NPS process have the right idea. The missing link is that they are moving forward without doing the basic research Bain did in deciding what question is the right one to use. It is not random and it should not be based on what seems to make sense. The selection of the question is foundational to the success of the effort. The more willing an alumnus/a is to speak about loyalty or relationship to their alma mater, the more likely they are to give in the future. While this is also true about their willingness to promote the university, the connection is not nearly as strong.
We recommend that, as part of a NLS strategy, you ask the following question after all events and in your larger alumni survey projects:
“On a scale of 0 to 10, with zero being the lowest, how would you rate your loyalty to XYZ University?”
If you also want to include a question about event attendance, we recommend following the six-question USC model that Scott Mory presented at the 2013 CASE Summit (see below). The recommend (or promote) question is appropriate when measuring the more transactional nature of event attendance. In using Scott’s questions, we recommend that question 5 should be based on loyalty and not willingness to promote.
THE BASIC NPS QUESTIONS
NET PROMOTER SCORE | 20
-
- Based on your experience at this event, how likely are you to recommend a similar event to another Trojan? (0 to 10 scale)
- Would you please share why you chose that number?
- Did this event meet or surpass your expectations? (1 to 4 scale)
- Anything else you would like us to know about your experience?
- Based on your experience at this event, how likely are you to promote or support USC? (0 to 10 scale)
- Contact info for follow-up
We recommend changing question 5 to: “On a scale of 0 to 10 with zero being the lowest, how would you rate your loyalty to USC?”
If you implement a model like the one Scott presented at the CASE 2013 Summit, which he modeled after the Stanford and Cornell model, we would recommend changing question 5. Based on our research, this question needs to be tied to loyalty to or relationship with the university. Movement in this metric offers the greatest opportunity for movement in alumni intent to financially participate with their alma mater.
Now, what is the right way to score the NLS? If university philanthropic programs are just like low-cost airlines and rental car companies, then we would automatically conclude the Bain approach of using the Net Score, derived by subtraction low scores from high scores, is appropriate. Now that we have the right question, our next post will address this question of finding the right way to score the NLS.
Blog written by Robert D. Shoss
Analytical support provided by Jonathan Gaines