What we talk about when we talk about “experimentation”
By Ehsan Gul, Head of Experimentation, UNDP Accelerator Lab Pakistan
This blog is the second installment in a three-part series taking an in-depth look at the new ways of working within the UNDP Accelerator Labs. Check out the “what we talk about” series on Exploration and Solutions Mapping.
What pops in your head when you read the word ‘experiment’?
Chemical being poured in a test tube, apple falling on Newton’s head or a lady in a lab coat working with bacteria? You are not entirely off track here.
An experiment is simply a procedure to test an idea or a ‘what-if’ question. Here at UNDP Accelerator Labs in Pakistan, where I work as “Head of Experimentation,” we feel that the word has so much more to offer — well beyond the confirmation of an idea. What I have learned is the power of framing and solving problems around us as an experiment. When we’re experimenting, we become interested in knowing and exploring the “unobvious” or invisible around us.
Experimentation helps you explore the unobvious space
In my role as the Head of Experimentation here in Pakistan, my job is to help institute rapid learning about emerging development challenges through the design and running of a portfolio of experiments that is coherent with the needs of the local community, government and UNDP country office. Experimenters like myself working for the Labs are creating multiple intervention points to help address complex challenges.
In practice, this means that my team and I run multiple experiments at the same time and attempt to learn from them at the systems level. Experimentation can be highly valuable as it can break down big issues into smaller questions, which can then be more manageably investigated, in a way that is structured and transparent. It’s safe to say that for us working at the Labs, we do not see ideas as solutions, but rather as testable hypotheses. The experimentation process helps us find out what works and what doesn’t.
I find the Labs’ approach for developing innovative, locally sourced solutions to be an exciting opportunity to engage more deeply with grassroot communities in Pakistan. I have always believed that meaningful solutions to the world’s toughest challenges must come from the ground up, involve collaboration, and be innovative — maybe even audacious. For me as an experimenter, it is all about being an ideas person.
“Innovation amateurs talk good ideas; innovation experts talk testable hypotheses.” -Michael Schrage
For the Heads of Experimentation working in 60 countries across our network, experiments teach us to take a different attitude, summed up in our ‘test, learn, adapt’ mantra. In this way, we embrace failure as intelligent, structured and mitigated — although not always without discomfort. The world of experiments can be tough to navigate. A wise approach is to be open to the breadth of methods, while also thinking carefully about which ones are best suited to getting the answers we need.
Experimentation and Uncertainty
Often the environment we operate in is increasingly complex and uncertain, the issues we are trying to tackle are too complicated for a linear process, and there are no ready-made solutions available. The traditional plan-prepare-execute approach is not enough, but we need other tools to deal with complexity and uncertainty. Experiments bring tangible evidence early in the process when it is still possible to change the direction without big costs. Therefore, as shown in the figure below, the earlier we start experimenting and collecting information, the quicker we can reduce the level of uncertainty.
Experimentation to tackle plastic waste in Pakistan
Our process starts with acknowledging that ‘we don’t know’ even if it makes us uncomfortable. We need to be curious to constantly question and challenge our own assumptions.
Let’s take the example of plastic waste in Pakistan. Using Nesta’s decision tree for running experiments, our team of three (including Head of Solutions Mapping and Head of Exploration) reflected and set our direction for developing a portfolio of experiments. The yellow highlighted bits in the diagram below shows where we started our journey to tackle this wicked problem.
As part of our first learning cycle, we are deploying multiple techniques to identify hypotheses to put our bets against. Few of the techniques we are using include:
● Capturing citizen insights through firsthand interviews and discussions,
● SolutionsFest to bring designer community together to identify possible prototypes by our Solutions Mapper,
● Supporting innovation challenges — for example, finding workable hypothesis through this year’s Gen Unlimited Pakistan ‘Opening Doors, Knowledge and skills for Empowered Youth’ challenge.
Building on these initial steps, our Solutions Mapper recently conducted a 3-day collective intelligence workshop with Unilever at their factory site in Rahim Yar Khan (RYK) along with our Environment and Climate Change Unit for generating potential ideas for making the city plastic pollution free, and many such techniques to explore new frames and actively pursuing signals of change.
Spreading our bets in a portfolio of experiments
Moving past the exploration stage, we are designing multi-factorial experiments to test hypotheses and leads identifies by our Solutions Mapper addressing the issue in a portfolio manner. In easy terms, that means putting the problem in the center and throwing experiment bets across multiple factors to generate systemic change, rather than taking a one-solution approach.
Our Lab is working extensively with UNDP Pakistan units and programs, along with external nontraditional partners including waste management companies, private manufacturers, thinktanks, environmental watchdogs and so on, to maximize the impact and promoting a holistic approach.
Creating multiple intervention points to shift the complex system
A portfolio is a set of connected innovations that learn from each other: a deliberate set of connected experiments (Nesta, The Experimenter’s Inventory, 2020). The key word here is connected.
Let me walk you through how this is done in practice through a real-life example from our work at Accelerator Lab in Pakistan. We have identified ‘plastic waste’ as a frontier challenge for the Lab. In an effort to build momentum towards a plastics system that works, we are working with local innovators, industry and government to develop systemic solutions to promote the transition towards a circular economy for plastics in which they never become waste or their leakage into the environment is minimized. Learning from our sister lab in UNDP Philippines, we were inspired by their experimenter’s design canvas which helps put each experiment into an action plan and understand it’s spread across the portfolio. The following is an example of the canvas which we just started filling (note: this is not the final hypothesis but just an illustration).
What are we getting out of experimentation for development?
As UNDP is working towards reimagining development, we are moving towards putting learning at the core of their programming. Undoubtedly, experimentation is one of the major added values from UNDP’s Accelerator Labs at this stage. Experimentation is all about learning through gathering data, answering questions and testing assumptions. Experimentation can help us navigate in the avoidable uncertainty that is part of any innovation process, by remaining curious. Learn more about what success looks like for the Accelerator Labs Network at UNDP and beyond.
Curiosity is catchy! Stay curious. Never be afraid to test!
Author’s Note: The lab has three core functions i.e. exploration, solutions mapping and experimentation. All three functions contribute towards the portfolio. The work mentioned in this blog has greatly been informed by the work of our Head of Solutions Mapping, Ms. Javeria Masood.
Editor’s Note: This blog on Experimentation is the second installment in a three-part series, taking an in-depth look at the three roles we have at the UNDP Accelerator Labs. You can read the first one on Exploration here.
