Organizations spend billions each year on leadership development efforts but tend to overlook the important next step of demonstrating impact of their investments or ROI (“return on investment”). According to a 2018 report by Development Dimensions International, only about one in five organizations (18%) gather and leverage metrics relevant to demonstrating impact (DDI Global Leadership Forecast, 2018). So, you might be wondering: Why?
Having worked over the years with organizations of varying sizes, industries, and data sophistication, I have come to realize that there are many reasons for why organizations overlook the step of demonstrating ROI. Some of those reasons include (a) uncertainty about what to measure; (b) lack of clarity around what should impact what; (c) difficulty in collecting the right kind of data; (d) challenges surrounding the ability to pull together disparate datasets; and (e) lack of systems and processes to take action. All these reasons can be summed up to one general theme of “How?” My goal for this piece is to share three simple steps that most, if not all, organizations can take toward demonstrating impact.
When people talk about demonstrating impact or showing ROI of leadership development efforts, there is an implicit assumption that this term needs to be followed by an impressive dollar amount. While this certainly is a worthwhile goal, and there is great value to measuring and linking behavior changes and mindset shifts to meaningful business outcomes (see CCL’s Leadership Development Impact [LDI] Framework), what one needs to remember is that demonstrating impact of a given leadership development program should be viewed as an ongoing process requiring continued modifications over time. The ability to determine what is working and what is not working is one of the core benefits of impact analysis and a key to maximizing ROI. Insights gained through this process should be used to continually modify program elements to optimize impact and ultimately demonstrate impressive ROI.
So, how can an organization get started? Below are three simple steps that organizations can take to start their journey of demonstrating impact of their leadership development efforts.
- Identify the goals of the leadership development program to determine the metrics.
To help identify the relevant metric or set of metrics on which to focus for your impact analysis, start by examining the goals of the program. What is the program designed to achieve? Is the goal to raise the level of a leader’s emotional intelligence? Improve communication skills? Learning agility? Focusing on the goals of the program can help determine what outcome(s) you might expect if the program was successful in delivering on its promise.
An important caveat to note during the design phase, which has implications for impact analysis is that the more abstract the concept being taught (e.g., emotional intelligence vs. improving communication skills), the more rigor one should apply in operationalizing the concept. Operationalization here means that you need to not only clearly define the concept, but you also determine how the concept will be measured. To take emotional intelligence as an example, you might start by asking the following questions: (a) What do we mean by emotional intelligence? (b) Is it a single, broad idea, or is it made up of multiple, smaller concepts? (c) How will we measure or quantify this concept? and lastly, (d) Is there clear evidence that our current instructional approach will lead to improved emotional intelligence?
With respect to the impact of the program, if the program is successful in raising the level of the leader’s emotional intelligence, we might expect this program to have – over time – a positive impact on a number of outcomes such as manager engagement (i.e., the degree to which team members are motivated/inspired by their manager), absenteeism, turnover, organizational citizenship behaviors (i.e., constructive employee behaviors that fall outside of their job description), and even have the effect of reducing extreme withdrawal behaviors such as theft and sabotage.
- Determine if existing metrics can link to program goals.
Once you have identified a set of potential outcomes linked to program goals, the next step is to examine the metrics your organization currently gathers and maintains. Going back to the emotional intelligence example, if your program is successful, you might expect manager engagement to improve over time. For this metric, you may want to look to your organization’s annual engagement, climate, or culture survey. If your organization conducts employee surveys on a regular basis, does it contain a category or topic that can be used to infer engagement to the manager? It may be the case that your engagement survey vendor uses the label “supervision” or “leadership” to gauge the degree to which employees view their managers favorably. If so, these can be used as proxy metrics for manager engagement.
A reduction in turnover might also be a metric linked to your leader’s improved emotional intelligence. Employee turnover is a common metric maintained by most organizations but there are two caveats to using this metric. First, rather than using a single number (e.g., past 12 months), you will want to look for trends over time. This is because the impact of emotional intelligence (or lack thereof) is likely to be expressed and felt over longer periods than other interventions. Second, it is important to separate voluntary from involuntary turnover. For our example, the former should be used.
- Create new metrics to fill measurement gaps.
What if your organization collects data on turnover but nothing else? Since we know it will take time for emotional intelligence to impact turnover, what other metric or metrics might be helpful for gauging impact? Here, you are seeking to find new sources of data to link to program goals. Continuing with the emotional intelligence example, you might consider supplementing your turnover metric with such precursors to turnover such as absenteeism, and depending on your industry, tardiness.
If your organization is relatively small, a handful of focus groups or one-on-one interviews with direct reports (of participant leaders) can also reveal valuable information about impact. With focus groups and interviews, it may be useful to collect quantifiable data by incorporating a small number of rating questions directly tied to respondents’ thoughts on their managers’ emotional intelligence level. If your organization is too large to conduct focus groups and interviews, it may be time to consider conducting annual or bi-annual pulse surveys. Survey data can be a great source of impact data, especially when the topics align with program goals.
Conclusion
Despite the enormous investments made each year, many organizations overlook the importance and the value of gauging the impact of leadership development initiatives. While challenges exist, LD/OD professionals can begin by taking small steps toward demonstrating impact and ROI. By linking the specific goals of a development program with available and/or new metrics, one can uncover valuable information that can be used not only to inform ongoing refinements to the program but to demonstrate tangible impact expressed as ROI.
