Innovation Paradox
Organizations seeking better business results from innovation often confront an apparent paradox: On the one hand, they can point to an impressive track record of innovations that contributed to their current market position. On the other hand, their people are unable to explain how, precisely, this innovation success was achieved. This incomplete picture of how innovation really happens makes it difficult to determine what’s working well, how to replicate it or what could be improved.
The first step toward resolving this paradox requires recognition that, in most organizations, a great deal of innovation happens via informal mechanisms. Consider the case of an insurance company we worked with that had reached its top-tier market position, in part, by introducing a number of industry-shaping innovations over the years. One example they were particularly proud of was an early “mobile agent” app that enabled policy shoppers to interact via their phone with a sophisticated virtual agent powered by artificial intelligence.
This project started when a junior information technology (IT) developer, whose outside hobby was creating gaming apps, decided to spend time experimenting with a mobile agent app as a personal challenge. After developing a prototype, he shared the result with his manager, who was enthusiastic about the idea and encouraged him to continue. Shortly thereafter, the company chairperson visited their department, where he was shown a demo. The chairperson was so impressed, he pushed for it to be resourced with a project team, and from there, it followed the formal product development process. While the idea went through a series of well-defined stages all the way to launch, it was clear its genesis and early development — including early decisions to allocate time to it — were pure serendipity.
This is an example of an informal (or quasi-informal) approach to innovation that produced great results. However, these mechanisms just as often lead to wasted time and resources and, if not made explicit, are difficult to manage, replicate and scale. Moreover, it’s impossible to be certain the organization is creating all the innovation the enterprise strategy requires as there is no clear picture of how it’s done. In this case, the developer made a choice to go underground with the early stages of the development process (something we’ve seen with sufficient frequency in other organizations that we describe the pathway as an “underground tunnel” (see figure 4)). While this story had a happy ending, innovations that journey along the underground tunnel are far less likely to deliver successful outcomes.
In this e-book, we introduce the concept of an innovation pathway as a way to describe with precision how innovation really happens in an organization, whether it’s informal, formal or somewhere in between. Mapping current innovation pathways is an essential step toward getting more value out of innovation investments as it reveals:
- Bottlenecks in existing pathways that can be fixed to make them more effective.
- What is already working well and could be accelerated, standardized or expanded across the organization.
- Pathways that are poorly connected to strategysetting or resource-allocation mechanisms within the organization.
- Missing or incomplete pathways that need to be built out to support the organization’s strategy.
We will first define what we mean by an “innovation pathway” and share examples from companies we have worked with. Then we will explain how organizations can uncover their own pathways using an approach we call “pathway mapping.” Finally, we will suggest ways to use the resulting insights to get more value out of innovation efforts.
Introducing Innovation pathways
Like people, innovative ideas have their own life cycle, with stages analogous to birth, infancy, adolescence, maturity and even death (though some may achieve immortality). While the evolutionary details of different innovations may vary widely, there are four general stages that typically occur:
Stage 1: A problem worth solving is identified.
Stage 2: A possible solution is developed.
Stage 3: The solution is tested and refined.
Stage 4: The solution is scaled.
Of course, most formal innovation processes are more complex than this, but at a high level, they can all be mapped onto these four universal stages, each of which involves activities and resource allocation decisions. Because this is true for all innovations everywhere, we call this framework the “universal innovation pathway” (UIP):
An innovation pathway is the set of steps an innovation takes from initial idea to implementation. For a real-life pathway to be complete, it must include activities and resource allocation mechanisms that enable each of the four steps described by the UIP.
This picture might seem simple, but it has great value as both a diagnostic tool for understanding all kinds of pathways and as a design tool for developing the pathways the organization needs. Consider how it might be used to map the journey of the mobile agent app (fig. 2) previously discussed.
Not only does this chart illuminate the real-life mechanics that led to an important innovation (i.e., the mobile agent app), it also suggests a number of ways things could be improved. It shouldn’t have taken an underground pathway or a chance encounter with the chairperson to ensure the idea was pitched. This pathway might be strengthened by creating additional ways in which personal projects are encouraged and given airtime in front of senior leaders. Just as important, a project like this might have run into significant resistance given its potential for conflict with the existing business model (i.e., by creating channel confusion). To address this, the firm might consider taking preemptive steps to clearly communicate both the types of problems they believe are worth solving and potential solutions that might be “off the table” to provide guidance to would-be innovators.
Many organizations develop explicit pathways for different types of innovation. For example, Procter & Gamble, a company with a long legacy of innovation, identified four distinct types of innovation that were critical to its long-term success — sustaining, transformational, commercial and disruptive innovations — each of which was supported not only by a distinct set of processes but also by dedicated shared services (groups).
How do you discover and map your current pathways, especially if they are informal or invisible to most people?
We will discuss in more detail the types of insights that can be derived from a clear understanding of an organization’s innovation pathways. But first, we will answer a question that should logically come first: How do you discover and map your current pathways, especially if they are informal or invisible to most people?
Mapping Your Innovation Pathways
The good news is that all the information needed to comprehensively map out pathways already resides within each organization, though it may require a bit of digging to find it. We call this “innovation archeology,” or the study of the artifacts and evolution of past innovation projects, both those that have been successful and (just as importantly) those that were not. There are three steps involved in this approach:
- Identify past projects for review.
- Map the actual pathway taken by each project.
- Identify common pathway patterns.
Step 1: Identify past projects for review.
Pathway mapping starts by identifying a diverse range of historical innovation projects that will be the objects of study and which, ideally, would have evolved through a set of well-defined pathways. A good way to assemble this list is to ask leaders in different parts of the organization to provide examples of innovation projects that cover a representative range of characteristics. We typically recommend seeking diversity across the following four dimensions:
- Innovation type (e.g., core product innovations vs. transformational business model innovations, assuming these are not related in a 1-to-1 ratio to the defined pathways)
- Project size (e.g., major strategic investments vs. smaller, incremental innovations)
- Strategic objective (e.g., projects aimed at new growth vs. market share preservation vs. cost reduction)
- Project outcome (e.g., successes vs. projects that struggled or failed entirely)
The last dimension (project outcome) is perhaps the most important to the innovation archeologist. Projects are often swept under the corporate carpet because they failed to be successful — not because anyone is trying to hide them but because the project conclusion meant that teams were disbanded, assets repurposed and individuals reassigned. As such, these projects can be hard to identify, and it can be harder still to reengage the people who worked on them to share what went wrong (and what went right).
However, these less-than-successful projects often provide the most information about where pathways are suboptimal. They can provide important insights into the gray areas where formally defined pathways were insufficient to support the innovators or where systemic barriers to innovation wreaked havoc with the team’s ability to be successful. We would suggest that at least 30% of the projects an organization selects to review be those that fall into the “less-than-successful” category.
Step 2: Map the pathway for each project.
For each project, the organization should seek to understand and document how each of the four steps in the universal innovation pathway was completed. This can be done through interviews with people familiar with these projects, supplemented where possible with any existing documentation.
When constructing a pathway map, it’s helpful to triangulate on these observations by checking four separate sources: the project leader or key team members, the project sponsor, key project documents created during each of the four stages, and an impartial observer who had no stake in the project’s success or failure but may have a “water cooler” perspective on what really happened.
Stories are subject to many biases (which are critically important to understand), but documents alone fail to provide the important context and behind-the-scenes insights that often reveal the real reasons a project struggled to achieve success.
As organizations map out the history of each project, they should explore three categories of questions:
- What activities took place to accomplish each stage? This includes any activities performed by the team as well as key decisions that advanced the project.
- What resources were used (typically people and investment) to perform these activities, whether they were formally or informally assigned?
- What enablers and/or barriers did the team encounter as they moved through the stages? To what factors did the team attribute the project’s success or failure?
Figure 3 depicts a more detailed pathway map for the mobile agent project, summarizing the team’s observations in each of these three categories, as well as their initial ideas for how the pathway might be strengthened.
Step 3: Identify your common pathway patterns.
Once organizations complete the pathway mapping for the full set of projects under review, the next step is to assess any common pathway patterns.
Innovation projects can fail for many reasons, including the possibility that the idea was not a great one from the start. For this reason, interpreting the patterns across pathways is more complicated than simply determining which activities or enablers were present (or which barriers were missing) for successful ideas. It requires an assessment, in aggregate, of whether established pathways were appropriately resourced and whether they allowed teams to come to the right decisions about scaling up or shutting down as quickly and efficiently as possible.
Of course, if any individual project team missed a key activity or ran into barriers along its pathway, this isn’t necessarily an indication of a systemic issue, so when we conduct an analysis, we look for patterns across multiple projects within a defined pathway. For example:
- Activities: Are there common approaches to how and where opportunities were spotted, which teams spotted them, or how ideas were prioritized? Are there patterns in which types of activities were prioritized and in what sequence?
- Resources: Are there commonalities in how resources get allocated, where funding comes from or where people look for talent to support innovative projects?
- Enablers and barriers: Are there patterns in how shared service groups inadvertently slowed progress, where bureaucracy inhibited speed or created inefficiencies, or leaders who knocked down barriers? Are there common challenges cited by teams in the way their leaders interacted with the teams?
In figure 4, we share nine common pathway patterns present in many organizations. This list is not exhaustive but can serve as a useful diagnostic aid when looking for patterns that may exist in your organization.
These pathway patterns are caused by common failure modes that occur in many innovation systems. Some of these failure modes are caused by failures within the local governance of the pathway; others have root causes that lie elsewhere in the organization or at the intersection of other business units and shared service functions within the pathway.
For example, we see the dark pathway in many organizations that have a strong technology or engineering bias, and the resulting pathway fails to incorporate sufficient activity to build customer understanding early in the process. This pathway is often characterized by a team that makes rapid progress (for example, against building prototypes) and then discovers only later that they have built a solution without an urgent problem to solve. Addressing the issues resulting from a dark pathway requires clarity on the customer’s underlying job to be done — which should serve as the North Star for all innovation pathways.
Another common pattern is the silent pathway, in which a team — so confident in its assumptions about what it will take to be successful — fails to iteratively shape the idea based on input from key stakeholders (such as customers, partners, peers, leaders). Put another way, they are allowed to establish an operating model focused on execution versus discovery, and then discover too late they made too many flawed assumptions. The key is to develop a standard operating model focused on searching for scalable solutions, with the right cadence of internal debate fueled by market-facing activities to test assumptions.
These first two patterns are prime examples of local pathway breakdowns due to governance. Both are relatively easily addressed by the innovation team itself (with the support of its leader) because the issues lie within the pathway, and the means to address these issues are fully controlled by the innovation team.
Some pathway failure modes are systemic in nature, with the root cause of project failure lying in the interaction (or lack thereof) between the innovation team and other components of the innovation system.
In contrast, some pathway failure modes are systemic in nature, with the root cause of project failure lying in the interaction (or lack thereof) between the innovation team and other components of the innovation system.
The tollbooth highway is a good example of a systemic failure mode, where the team’s progress is hindered by well-intentioned supporting functions. This pathway is seen most often in highly regulated industries (e.g., financial services, pharmaceutical, medical devices, etc.) where supporting functions such as risk, global security and legal have all evolved to create multiple lines of defense against potential threats or against making mistakes that could cause significant harm (to customers, patients or the organization). These support functions are essential, but they are frequently built for rolling out ideas at scale and are ill-suited to handling the types of risks associated with the development and testing of innovative new products and services. Indeed, when they are asked to support the innovation team, they often do so by requiring the innovation to work within exactly the same interaction model they use with everyone else — with the result that the team’s ability to move quickly is reduced to near zero as they are forced to pause and wait until some other function completes its assigned task.
We recently worked with a large bank that was launching a new online product. The leader of this new product lamented to us that the size of the various governance and risk management teams assigned to oversee his efforts was four times the size of his actual project team.
Another systemic failure mode we call the paper trail, in which the leadership and governance model overseeing high-uncertainty pathways borrows too heavily from the core and treats projects as though they were execution-focused. For example, we worked with an innovation group in an enterprise software firm that complained they spent more than 80% of their time preparing update materials, financial pro formas and investment requests, and only 20% on actually making progress toward understanding the customer or developing the solution. Innovation teams must be given space to focus on advancing their discovery activities, and overly burdensome reporting requirements — particularly if they are focused on reporting on the wrong metrics (execution metrics vs. discovery metrics) — can make a bad situation worse.
A fifth systematic failure mode is the road to nowhere, where an innovation team is surprised to have a project shut down — right as they seek to step up their funding and enter into a pilot. We recently worked with a team in a global logistics provider who went through this exact experience. The team was excited to pitch their project to the executive team and begin a multi-month pilot with a customer. The CEO had heard of the project and had a rough idea of what they were proposing, but as he heard the details, he realized that this new service offering would compete with one of their existing offerings, and even worse — create confusion in the minds of some of their core customers.
The innovation team was confused — they had purposely designed this offering to be disruptive and to create a potentially new economic model that would be attractive to some of their customers. In hindsight, had the CEO realized what the team thought they were trying to build, he would have shut down the project at the beginning and saved the team six months of work.
We’ve seen variations of this story play out countless times over the years, and it highlights a systematic failure that exists at the intersection of innovation pathways and strategy. Addressing the issue requires a clear translation of the enterprise strategy into priorities for innovation and a clear linkage of these priorities to the corresponding pathways that support them.
The tollbooth highway, paper trail and road to nowhere are all examples of systemic issues that show up in the pathway patterns. These types of pathway patterns are usually harder to correct, as they require a systems lens to address — which may, in turn, require interventions in how strategy is set and communicated, how resources are allocated, or how employee performance is evaluated.
Unlocking Value From Your Innovation Pathways
The real value of pathway mapping comes not just from seeing how innovation is really happening but in using this knowledge to understand what’s working well and where there are opportunities to improve.
Unlocking the value from a pathway usually requires some combination of fixing the underlying pathway (e.g., adding or reordering activities along the pathway) as well as addressing systemic issues in the way that innovation priorities are set, innovation portfolios are handled, and people are managed.
After organizations have mapped their pathways and identified how they might be strengthened, they should also consider these four questions:
1. Are there missing pathways that should be created?
Sometimes, the mapping exercise can reveal the lack of a pathway the organization believes will be important. For example, one chief operating officer (COO) of a large global enterprise, which innovated in each of its local markets, lamented to us that one of his job functions was “connecting dots” as he would travel from market to market, and poke or prod leaders to learn about something he’d seen being developed by peers in another market. His wish was for a “global scaling pathway” in which ideas, identified in one market, could be more quickly uncovered and selected as suitable for scaling across multiple markets. To support this, his team used pathway mapping to understand the key barriers to scaling great ideas globally. For example, there was simply no standard idea transfer mechanism for ideas across regions. This was exacerbated by a siloed organizational structure and a culture with a distinct “not invented here” mentality. With these insights, the team was able to design a series of quick interventions to more effectively identify and push promising ideas across regions.
To know if they have the right pathways, organizations must first clarify the innovation priorities the pathways should support. At the very least, there should be two distinct pathways in any organization:
- A low-uncertainty “product development” pathway that serves as the standardized process by which most core innovations will progress: These types of pathways often go by names such as the new product development (NPD) process, and are characterized by execution-focused management in which tasks can be clearly sequenced and managed on a timeline. There may be variations on this theme focused on costs or customer experience, with differing sets of activities or criteria to advance through stage gates.
- A high-uncertainty pathway along which those higher-risk ideas will develop: For example, the “step out” pathway was developed by a multinational chemical company to manage the uncertainty of introducing new innovations into its large chemical plants — changes that would require extensive testing, tweaking and training to get right. We have also seen numerous variations on the Lean innovation process, or custom pathways that were built to leverage the commonly used tools (e.g., the business model canvas). These high-uncertainty pathways are characterized by discovery-focused management, where the assumption-to-knowledge ratio is high, and activities cannot be managed against a timeline but instead must be managed according to the amount of uncertainty remaining.
2. Where are there bottlenecks or gaps in existing pathways that could be fixed to make them more effective?
With one client, in which there was a wonderfully strong engineering culture, there were a number of critical blind spots in some of the upfront stages of one of its pathways. The lack of customer insights and economic considerations would frequently lead to over-engineered proposals for new products or solutions, which might have been easily and quickly prevented by asking “does anyone want this?” or “would anyone pay to solve this problem?”
Another client discovered that there were several consistent bottlenecks that pertained to the lack of mechanisms to access small amounts of funding, and also struggled with the fact that it couldn’t reallocate people from one project to another due to complexities arising from the way that teams were staffed and managed — without wreaking havoc on departmental budgets.
To help identify and then address these bottlenecks, we recommend that organizations benchmark their existing pathways against best practices. There are many well-documented approaches that provide guidance on the logical sequencing of activities and examples of templates and tools.
3. What’s already working well and could be accelerated, standardized or expanded across the organization?
For example, in one organization, we found five independently defined pathways — all of which were designed to achieve the same result (new business models), all of which worked quite well, but all of which used their own tools (e.g., market sizing methodologies, business model canvases) and stage gate definitions, making it hard to compare project progress on an apples-to-apples basis to drive capital allocation.
As a general rule, we recommend that organizations embrace enterprise-level approaches to innovation (defined innovation types, clear innovation pathways, etc.) as this greatly reduces the challenges associated with investment planning and capital allocation when leaders can compare and contrast competing investment opportunities from disparate parts of the organization. Similarly, the impetus to standardize, share and scale can often only come from the enterprise, which is best positioned to identify and address systemic issues that occur across regions and business units.
4. Where are pathways poorly connected to the strategy-setting and resource-allocation mechanisms within the organization?
In one of our recent engagements, we spent time with the innovation team, which consisted of an interesting mix of strategists and entrepreneurs, many of whom had come from different companies and industries. This team spent a great deal of time in the field talking to customers and could be counted on to have an up-to-date sense of what was going on in the periphery of their core markets (e.g., which startups were getting funded, what customers were saying about the future of the industry, etc.).
Yet, we were surprised to find that few of these insights ever made it to the ears of the executives making the consequential decisions about resource allocation to drive growth.
These critical insights must flow in both directions. Strategy must be informed by pathways activity (e.g., customer insights), and pathways must be connected to strategy (by strategic focus areas). To address the latter, organizations must ensure that they point their pathways at only those opportunities that are strategically consistent with the vision the organization has for how and where it will choose to compete in the future and where the leadership team is committed to invest.
Benchmarking to Best Practice
Much is known, and a great deal has been written, about best practices for innovation within any of the UIP steps described above, and after an organization has mapped out the actual steps of their pathways, it’s a great idea to benchmark against these best practices.
In this e-book we will not go into detail on the specifics of best practice for each pathway, except to note that you need to have a clear vision for what an effective pathway should look like and the underlying principles that should guide its design (e.g., customercentric, hypothesis-driven, learning-oriented, etc.). Figure 5 shows an illustrative example that highlights Innosight’s perspective on what key activities, metrics and exit criteria might be used in an effective pathway for high-uncertainty (e.g., new business model) innovation.
With clarity on best practices, organizations can comprehensively evaluate any variations in their own pathways and easily identify where key activities were neglected (or missed altogether). Uncovering the hidden innovation pathways is a necessary first step to unlocking the latent value lying dormant in each organization.
The innovation pathways mapping exercise can be a powerful tool for executives seeking to understand how innovation really gets done today and offers a strong foundation on which to prioritize the various interventions they might pursue to strengthen their ability to more systematically drive growth through innovation.