In previous blogs, we went over 1) how one might gauge the impact of leadership development efforts by using existing or new measures after a program has been rolled out, and 2) how one can leverage existing or new data to pinpoint potential areas of leadership development for a given target leader population. This blog goes into the next level of quantifying ROI of behavior change. We will review a real-life on-going case study which uses a two-phase discovery approach to gain insights used to inform the program design. The discovery phase will be followed by a planned post-program survey to examine the behavior changes in the target leaders as well as estimate the impact (i.e., ROI) of those behavior changes on critical business outcomes.
Background
This real-life example involves a well-known non-profit organization in the healthcare space which sought to design and execute a 6- to 9-month leadership journey for their top executives (i.e., target leaders). In order to inform the development solutions’ design, we proposed and executed a Discovery process involving both qualitative and quantitative elements. The qualitative component was completed using executive interviews while the quantitative component was completed using CCL Fusion’s LCS90, a 10-minute survey administered to direct reports of target leaders. In addition, and rightly so, this non-profit client sought to incorporate a purposeful measurement system that would enable them to estimate the impact of the leadership development program on key business outcomes.
The two-phase Discovery approach used is briefly described below.
Discovery Phase 1: Executive Interviews
The first phase consisted of discovery interviews with the CEO and five members of the Executive Team (ET). The purpose of the discovery interviews was to identify and focus on high-priority competencies to address, target development efforts, and drive results that matter and last. Questions for the interviews were crafted to address inquiries around (a) major opportunities over the next four years; (b) top 2 to 3 most important outcomes to achieve as an organization; (c) major challenges over the next four years; and (d) client organization’s strategy with respect to these challenges, etc. To provide an example of the richness of these interviews, on the question related to addressing future challenges, one executive commented “We need a strategy to address our talent gaps across the board – what I’ve learned is that even though we are not a 100,000 employee organization we have the same needs for training & development that many organizations have…” Another executive stated “We must be more open to change; work across team boundaries; address conflict. Rather than being passive aggressive, we need to be more direct; that’s the only way to solve complex problems.”
Through the interviews, we also analyzed this non-profit’s stakeholder needs, assessed the delivery environment, developed a shared view of the organizational context, and determined the competencies and capabilities required for success.
An analysis of the interview responses revealed six key areas to which the organization should attend: (a) Leading change & transition; (b) Vision – communicating, executing & commitment; (c) Building high functioning teams; (d) Trust; (e) Accountability; and (f) Leading organizational culture change. These insights – gathered through qualitative means – were combined with a quantitative approach taken during Discovery Phase 2, described below.
Discovery Phase 2: CCL Fusion (via LCS90)
The second phase consisted of a quantitative component of Discovery involving predictive analytics by using CCL Fusion. Data were collected from direct reports of target leaders using CCL’s Leadership Capability Scan (LCS90). LCS90 is a 10-minute survey containing the following three sections: Section 1 – direct reports rate their managers (i.e., target leaders) on 20 competencies (16 CCL defined competencies and 4 competencies specified by the client) ; Section 2 – direct reports share their level of engagement with respect to their job, manager, team, and organization, as well as their intentions to leave their organization; and finally, Section 3 – direct reports rate their managers on four dimensions of leadership effectiveness.
As shown in Figure 1 below, our analysis of the Phase 2 data revealed 7 competencies (i.e., skills, behaviors, and perspectives) out of 20 to be most impactful on employee engagement and intentions to leave. These include: (a) Communication; (b) Trust; (c) Strategic planning; (d) Accountability; (e) Influence; (f) Leading change; and (g) Change management.
One might notice that 5 of the 6 areas identified through the qualitative Discovery were captured through the quantitative Discovery approach. This convergence in the areas identified for development further validated the results from the Discovery process and confidence in the development efforts.
Figure 1: Seven critical competencies driving employee engagement & intent to leave
Program Design
The two-phase Discovery approach uncovered useful insights towards the design of this non-profit’s leadership development program. Specifically, the program focused on skills, behaviors, and perspectives linked to those challenges identified through the qualitative and quantitative Discovery. Assessments and development exercises were also chosen and customized to create powerful experiences with the aim of moving the needle on those competencies.
Next Steps – Impact & ROI Estimation
Despite a slight delay in program execution due to the COVID pandemic, this non-profit’s leadership development program has recently resumed and is on scheduled to be completed by Summer of 2021.
Given the delay, the current plan is to re-administer (i.e., Time 2 administration) the LCS90 in Summer of 2022 to the same population (direct reports of target leaders). The results of the Time 2 survey will enable us to estimate (a) changes in the target leader groups’ behaviors; (b) examine shifts in those direct reports’ level of employee engagement; (c) examine changes, at the collective level, to the proportion of employees (i.e., direct reports) intending to leave the organization; and most critically, and (d) estimate the degree to which the changes in the target leaders’ behaviors impacted employee engagement and intentions to leave. The above changes will then ultimately be used to accurately estimate the ROI of our clients’ leadership development efforts.
Conclusion
While previous blog postings focused on ways to gauge the impact of leadership development efforts post-program (i.e., after program roll-out) as well as how one might leverage data to pinpoint areas for development, this blog describes how those two approaches (Discovery and Impact) can be incorporated into a single intervention. By taking a combined – qualitative and quantitative – Discovery approach, we were able to convincingly uncover potential areas of leadership development among more than a dozen competencies. This approach enabled this non-profit client to focus its investment dollars and development efforts on a subset of those competencies empirically shown to matter. By re-administering the same survey at Time 2, we will be able to uncover objective changes in leaders’ behaviors and, in turn, estimate the ROI of those changes on important business outcomes.
By taking a thoughtful approach to Discovery – one that combines both qualitative and quantitative elements, rather than relying on what “appears” to be the right set of skills and behaviors, organizations can gain valuable insights into what those “right skills” are for further development (see also Jeong et al., 2020; Young et al., 2018). In turn, by focusing their development efforts on those right skills they can maximize the likelihood of getting real results and ultimately showcasing the return on their leadership development investments (see Stawiski et al., 2020).
References
Jeong, S., Young, S., & Flink, C. (2020). Using analytics to prioritize leadership development initiatives. Training Industry Magazine. https://trainingindustry.com/magazine/jul-aug-2020/usinganalytics-to-prioritize-leadership-development-initiatives/.
Stawiski, S., Jeong, S., & Champion, H. (2020). Leadership Development Impact (LDI) Framework [White paper]. Center for Creative Leadership. https://cclinnovation.org/wp-content/uploads/2020/11/evaluationframeworkldi.pdf
Young, S., Champion, H., Stawiski, S., & Smith, M. (2018). Using Predictive Analytics for Leadership Development [White paper]. Center for Creative Leadership. https://www.ccl.org/wp-content/uploads/2018/09/using-predictive-analytics-for-leadership-development.pdf
