The Realm of Intelligent Failure

ROTMAN Magazine

Winter 2014

Point of View: Theodore Forbath

FAILURE IS AN INEVITABLE outcome of operating in an uncertain environment, which is why so many attempts at innovation go awry these days. However, some of the most successful organizations have embraced this fact — and the opportunity to learn from failure, thereby improving their innovation track records. 

Leading organizations, from Apple to Google and beyond, have learned from early-stage failures. A classic example is Apple’s Newton PDA tablet in the 1990s, a flop whose core concept eventually found its way into the iPad.  Or the predictable user furor over nearly any new Facebook feature, which eventually gets re-cast by the company as ‘market research’ or ‘design validation’. 

Learning from failure isn’t a new idea. A decade ago, when Henry Chesbrough of the Haas School of Business at UC Berkeley wrote his groundbreaking book Open Innovation, he showed that corporate failure can lead to fresh opportunities. Still, in today’s business culture — where the predominant goal is often to avoid failure and reward success — it can be difficult to view failure as something that adds value. 

A new breed of organization is emerging that embraces failure as an actual goal. For example, the Generic Infusion Pump project, a collaborative effort between the University of Pennsylvania and the Food and Drug Administration, recently designed a drug-delivery system ‘in reverse’: they initially concentrated on brainstorming as many potential failures as possible, and then pivoted to designing ways to avoid them.

Harvard Business School Professor Amy Edmondson — one of the world’s leading researchers on the topic of organizational failure — categorizes it into three types:

• Preventable,

• Complexity-related, and

• Intelligent.

In this essay I will focus on intelligent failure and show that it is an essential component of the R&D process.

Experimentation for the purpose of learning and innovating — a mandate central to the concept of intelligent failure — allows for the intentional allocation of resources for exploring product and service opportunities. Failure in such cases can be seen as an acceptable by-product of innovation, and businesses that are able to quickly assess and learn from these failures will be more innovative than those that don’t.

Intelligent failure was originally described by Sim Sitkin of Duke University as a process akin to the scientific method. Other academics are finding it to be a key element in the innovation process. Edward Hess and Jeanne Liedtka, professors at the Darden School of Business at the University of Virginia, report that in their combined 17 years of innovation research, organizations known for their successful inventions understand that failures are a necessity “up to 90 per cent of the time.”

In most organizations today, various facets of the product development processes manifest the concept of intelligent failure. However, they are often re-contextualized outside the sphere of failure as ‘pilot product launches’, ‘beta software releases’ or ‘customer feedback sessions’.  For its part, the company I work for [frog] has embraced the concept of ‘intelligent design’, frequently using rapid prototyping as a mechanism for validating product or service concepts so that failures can be identified and addressed iteratively and early in the product definition process.

Recently, we have been rapidly prototyping new mobile money services in Africa and Asia. This process involves bringing an interdisciplinary team of strategists, user/design researchers, designers and technologists into the field in the earliest part — the generative stage — of the innovation process.  The team uses ethnographic and immersive research methods to gain insights about user needs, behaviours and motivations that inspire new ideas. These are translated into low-fidelity prototypes that can then be evaluated with endusers.  With an interdisciplinary team, this process of insight generation, concept ideation, prototyping and evaluation is transformed from a sequential process into an iterative process repeated on rapid, daily cycles.

The key problem in encouraging intelligent failure is this: human nature is highly averse to the feelings of shame, embarrassment and low self-worth associated with failure.  Employees also often fear repercussions to their salary and status within the organization. As a result, common reactions are to deny failure or point fingers and blame others.  Based on my work at frog, following are several recommendations for overcoming the psychological barriers to risk-taking in your organization.

1. TRAIN, COACH AND MODEL ‘MANAGED FAILURE’. Organizations are often hesitant to institutionalize acceptance of so-called failures due to fear of creating an environment that doesn’t recognize the value of resources invested, or worse, a culture that doesn’t value ‘winning’.   Experiments that build on the concept of ‘managed failure’ can shift the mindset to accept that failed projects offer learning opportunities and are part of an iterative process.

Generally speaking, early-stage innovation experiments are activities planned to confirm a hypothesis using modest resources. The purpose is to gather new information to analyze, interpret and apply toward iteration and retesting.  Therefore, the experiments should be designed to represent true-to-life or representative environments, customers and materials with limited external influences. The purpose is to discover anything that could go wrong, such as issues in usability, safety, set up and customer service, before further development or commercialization investments are made.

During experiments, leaders can coach their team members by asking probing questions about the reasoning behind setup, execution and analysis. In addition, leaders can model ‘managed failure’ by testing an idea themselves and communicating results to the company, welcoming feedback and new ideas for iteration.

At Facebook, for instance, senior leadership — including CEO Mark Zuckerberg — tests product designs to make sure that they understand what elements of Facebook software and user experiences are delivering meaningful value. Engineers and designers do not merely present their findings to the C-suite; the C-suite engages in the iteration (and failure) process directly, to better understand it and make more informed executive decisions.

2. ALIGN PERFORMANCE MEASUREMENTS. To change the mindset around failure, leaders must align performance measurements with the acceptance of failure. People will be more motivated to experiment if they are supported by their organization through raises, bonuses, promotions and other career opportunities. During performance reviews, employees in innovation functions should be evaluated on the effective planning, execution, analysis, communication and application of learning from their experiments. For entire divisions dedicated to innovation, performance goals can be set based on the percentage of division revenue that is generated by new products launches, as well as the broader impact on the evolution of best practices, intellectual property and learning across the company. 

3. EMPHASIZE TEAMWORK. Innovation experiments can be stressful and emotional due to their ambiguous nature. Emphasizing the teamwork aspect encourages shared credit as well as shared learning opportunities. Toyota, for example, promotes team bonding through collaborative exercises and rewards entire teams (rather than individuals) who submit improvement ideas and take risks toward change. Team members also provide support and encouragement to each other to mitigate feelings of negative self-worth by documenting new insights and best practices. It is often important for project leaders to remind their team members that the purpose of an innovation initiative is as much about data gathering and analysis as it is about achieving success.

4. PUBLICALLY EMBRACE FAILURE. Company leaders can reinforce acceptance of failure by publically celebrating projects that didn’t quite meet expected results, but that were successful in providing new learning. Some companies, like Eli Lilly, even throw parties to celebrate their biggest failure, that is, the project that involved unknown outcomes and did not meet expected results but led to the greatest learning for the company. This type of public announcement is an easy way to communicate to the entire organization the value of managed failure.

5. ENCOURAGE EXPERIMENTATION WITH UNKNOWN RESULTS. One way to encourage experimentation across the organization is to set aside time within employee schedules. LinkedIn, for example, sets aside one Friday a month, called ‘hackday’, for employees to work on projects of their choosing.  They have even gone so far as to develop a program called [in]cubator that allows employees to pitch an idea as a group to senior leadership. If they get the green light, this team is afforded 30 to 90 days to iterate the idea and develop a working business model.

At frog, leaders allocate time for employees to explore the possibilities of emerging technologies. Most recently, we held an internal, company-wide competition to conceptualize wearable products that leverage the abundance of data available to consumers to heighten their everyday experiences.

The concepts ranged from an interactive friendship bracelet that allows users to record, relive and share memories, to a pollution mask that directs users to better pockets of air.

While our wearable competition did not result in the development of any new products, the exercise was valuable.  The concepts forced us to consider the design challenges inherent in low-fi, low-cognition interfaces that many wearable devices will rely on. The knowledge gained from these innovation experiments has given frog a head start on developing design principles for emerging, wearable computing devices that are predicted to explode in usage in the near future.

Another method of experimentation is to surface prior failures and then ask employees to re-evaluate them for potential solutions. Employees can conduct small information gathering sessions to identify what didn’t work the first time, and then test potential new solutions. Pfizer’s Viagra, for instance, was first created to relieve blood pressure, and although it failed in initial market trials, it became one of the most successful failures in history thanks to its ‘side effects’.

6. CONDUCT POST-EXPERIMENT FEEDBACK SESSIONS. After a failed project is completed or halted, team members can analyze quantitative findings through any appropriate statistical program, and then discuss possible changes to processes or assumptions based on the findings. For example, after any mission or training event in the U.S. military, a leader conducts an After Action Review. The group involved in the mission reviews what went well and should be sustained, what didn’t go well, what was learned, what can be improved upon and what the action plan is for making changes.  Everyone gets the chance to speak up and provide either general or specific feedback. Even in these high-stress situations, feedback rules are enforced, including withholding finger pointing. If the description of an event does involve an individual’s specific actions, phrasing such as ‘in this instance, this wasn’t the right decision’ is used.

At frog, we have adopted a similar approach and implemented an end-of-project retrospective framework to rapidly assess and learn from failures and successes. This process includes not only project team members, but also client representatives, so that multiple perspectives are reflected and results are shared across organizations. This approach has enabled frog and our clients to develop a shared culture that is accepting of managed failure and that encourages ongoing innovation.

7. LOOK FOR ADDITIONAL APPLICATIONS OF FINDINGS. Companies may gain an advantage by analyzing findings from innovation experiments for strategic value that goes beyond the initial project scope. For example, frog worked with a major technology company on prototyping and creating a reference design for a Home Energy Management System (HEMS). This was subsequently described as both a failure within the client’s organization (because it did not result in a product that came to market), and a strategic opportunity to gain early insights into emerging markets where the client could define new, industry-specific solutions for an emerging category of network-connected appliances in the home.

The HEMS project champions moved into new leadership roles, and their early insights into the energy management industry helped the company re-think its long-term vision for a new business unit that was mandated to gain an early beachhead in the machine-to-machine and intelligent systems markets. The project leaders formed a new Industry Solutions Group to advise on industry-specific strategies and solutions, rather than on products. This was only possible because the client’s corporate culture values the willingness of its colleagues to experiment, fail and use their experiences to influence the strategic direction of the organization.

8. SHARE RESULTS. After analysis, results of experiments should be distributed across the company to inspire others to apply leanings and further experiment with the findings.  In the U.S. military, results of after-action reports (AARs) are recorded in Leader Books and escalated through the ranks if needed. Feedback from AARs is analyzed, and trends are reported across the military through   publications called Lessons Learned reports.

In one instance, the military studied potential preventable military deaths and found that some wounded soldiers died because of extremity injuries and bleeding. As a result, tourniquets were introduced into the field and medics were taught how to use them. The military then went back to study this potential solution, and found a significant drop in preventable deaths with no ill affect from the use of the tourniquet. In share-out documentation, it is important to state not only what was learned, but also the recommended actions for change and future testing.

In closing

To overcome the various psychological and organizational barriers to embracing failure, managers can adopt the methods detailed in this article and, in doing so, re-cast ‘failure’ as something much more positive: progressive experimentation.

In the end, progressive experiments allow an organization to advance towards innovative marketable products or services in manageable increments that enable rapid course corrections at a lower cost. What leader wouldn’t want to encourage more of that?

frog partnered with Visa to develop a portfolio of innovative

financial services that better meet the needs of base-of-thepyramid

consumers in Rwanda. frog researchers spent over

1,000 hours in the field learning about consumers’ needs,

behaviours and motivations. In this image, a researcher is using

a mobile phone prototype rapidly mocked up the previous day

to probe a respondent’s attitudes towards trust and loyalty.

AirWave is one of 12 wearable computing prototypes that

frog created in 2012 through an internal Studio competition

program. Airwave is intended to be a connected face mask

protecting people from air pollution, while collecting location

specific air-pollution data at the same time. It then feeds

the pollution data into a smart phone application to visualize

and share it with a network of like-minded people.

Theodore Forbath is the Global Vice President of Innovation Strategy at frog, based in Cambridge, Massachusetts.

Share this post