Experiment! But Why?!
/“Let’s Experiment!” I heard one leader cry. “But Why?” retorts another.
The team was looking to improve the way they did things. The question was really how much effort should be invested into trying new things without any guaranteed payback.
The views of the two leaders reflect two different mindsets when it comes to managing problems or considering improvements. Some people seem naturally drawn to the idea of experimenting: trying new things, testing things out, curious to see if there is a better way. Their view is, “it kind of works, but perhaps it could be better”.
Others prefer things as they are: predictable and workable. Their view is, “tweak things if need be but if it ain’t broke, don’t fix it”.
I have been involved in a large variety of experiments – some producing ground-breaking results, some producing small incremental change and some producing nothing but new knowledge (a polite way of saying: 'what we thought we could do, we couldn’t).
And to me, both viewpoints hold some weight. I’ve seen good experiments and bad ones, so let’s look at why maybe we should experiment more, why we should be mindful of our investments of time and effort and what we need to do to ensure there is a decent return on our efforts.
Let’s Experiment!
Experimentation in its most basic form is an attempt to test or validate our ideas. As technology shifts, as structures within the organisation change and user demands shift, the problems we are trying to solve also change. Maybe we find ourselves trying to do more with less, trying to keep pace with a competitor or even keep one step ahead. In all these cases, we will need to adapt and try new things and experimenting can be a systematic way of testing our assumptions and documenting our findings.
But Why?
Whilst some will be convinced by the case for creating experiments and have seen the benefits, others will have witnessed poor experimentation, and let’s face it, there is a lot out there!
On the back of that, they may come to a very different conclusion: Experimentation is an investment of valuable time which often fails to produce a recognisable outcome. It is waste. It is a distraction from what we already do well. People who have been burned and frustrated by multiple failed experiments happening around them could well argue:
"Some people just like the idea of experimenting in order to chase rainbows but it's nothing more than a vanity project. Whilst they may argue that they are doing it in a very defined way, all that demonstrates is they are systematically chasing rainbows without a measurable goal.”
Bridging the Experimentation divide:
The argument above is really about experiments being run poorly as opposed to experiments being unnecessary. If we are going to experiment, we do have to appreciate that experiments are an investment of time – time that we could be spending on delivery.
So here are the 7 key things that I believe can make experiments worthwhile:
Frame the experiment: This means being clear about the problem we are trying to solve and the working hypothesis we are looking to test. We also need to be clear what success looks like.
Fail to measure and you will fail hard: people talk about the key to experimentation is to ‘fail fast or fail hard’. This is misleading - experiments are only a failure when we don’t measure. Any measured result teaches us something. Conversely, even if you think you’ve discovered something interesting, you have nothing to show if you haven’t clearly measured things like cost of investment, rate of return, overall value and risk factors.
Contract with your stakeholders: we need to be clear with stakeholders what we need from them and what they can expect from us. To do this effectively we need to ‘bound’ the experiment – we need to agree a timeframe and resources and what we will do within that framework. We will need to agree with them that we will be transparent with our progress. In return we must be given the time we need to ensure the experiment is able to prove its worth. Lastly, when we do produce our findings, we need to be clear about what we have found. In particular, what have we learnt about the conditions under which our new way of doings things will work, and the conditions in which it won’t.
Don’t stop delivery if you don’t need to: carrying out experiments doesn’t mean we have to stop delivering but it may affect our rate of delivery in the short term. I often work with groups to build experimentation time into their cadence: dedicating small chunks of time to first breaking down the problem, then ideating and then trailing things. This approach ensures that we maintain a balance between delivering results in the here and now and investing in the future. It’s also easier for stakeholders to understand and sign up their support.
Keep things simple: often we think we are trying to solve one thing only to find there is a huge number of factors at play. Good experimenters break things down into smaller elements so that each can be analysed. We then need to think about the quickest and simplest way possible of testing our hypothesis. Similarly, any solutions we propose need to be uncomplicated, or people won’t be able to buy into them.
Innovate rather than invent: inventing something means you have discovered something truly new, like Dr Alexander Fleming returning from a holiday to find a mould growing on a Petri dish that seemed to repel some species of bacteria. That is rarely what we are trying to do. Most often we are looking to respond to specific customer demands or shifting market conditions. Innovation is as much about improving an existing situation as it is about creating something new. Rather than trying to create a Eureka moment, we are often better off looking at what others have already done and refining these ideas for our own needs – a process of ‘adoption and adaptation’. We can also gather from others what hasn’t worked, and why. Researching what is already out there is normally a far more productive starting point than trying to create something completely new (which is probably already out there anyway!)
Complete The Protocol: if you don't complete the experiment you don't gain anything – accept no results and no learning. Don’t just drop something because you don’t think its worked, or hide your results from stakeholders. By starting an experiment you have entered into a psychological contract with your stakeholders to see it through. Remember, any measured result is a good result. One subtlety here is when, for some reason, an ‘early exit’ is required. But even then, the conditions for leaving early should be captured in the experiment definition and agreed with stakeholders before exit.
If we do these 7 things well, experiments can be low risk and allow us to keep pushing the envelope on what’s possible. Conversely, if we don’t invest some of our time in adapting and changing, we will become extinct.
Dinosaur out!