After reading this page, if you only remember one thing, make it this:
We instinctively know that there are millions of ways a project can fail and thinking how to prevent them all is overwhelming. But it turns out they can all be categorised into one of two groups. HOW to do something problems or WHY you’re doing it problems. Once you understand this, there are logically two ways of approaching any problem, one that does the WHY followed by the HOW in sequence (Routine) and one that does WHY and HOW together in parallel (Novel).
Both are very different ways of thinking about your problem solving approach but it’s always safest to assume everything is Novel until proven otherwise.
There are Two Reasons for Project Failure:
1. You don’t know HOW to do something.
The obvious one that you might go to school for or to seek help from an adviser or mentor, someone who has done it before… You (or your team) don’t have the skills, the knowledge, the ability to achieve what you’re hoping to achieve. But it might also be that you’re a bad manager and struggle to get the right people to help you. Any of the following could constitute a problem with your How:
Lack of knowledge, skills, ability, tools, resources or technology
Poor leadership or management
Not enough time or money.
Whatever it is, you can get advice, take a course, read a book, find more people, get more practise, be more empathetic or simply work harder to overcome it.
2. You don’t know WHY you’re doing it.
The one usually overlooked. If you’re solving the wrong problem, it doesn’t matter how well you solve it. If you look up the Forbes Largest Product Failures none of them are technically bad, they all worked. But no one wanted them and the team should have known that much sooner in the process. Problems with Why might be caused by:
Trusting a bad project brief
Poor market research / a false understanding
Weak customer insights
Unconscious bias, silo thinking or design fixation
Short term development goals and targets
Wrong assumptions & incomplete data
Jumping to wrong conclusions
Fear of being wrong
Going on Google or reading management books won’t help you here. You have to “Get out of the building” as Steve Blank would say and study your potential customers yourself, over and over again.
If you’re solving the wrong problem, it doesn’t matter how well you solve it.
So what do we do about it?
Not failing because of a HOW reason is just part and parcel of learning any skill or activity. We all make mistakes when we’re learning how to do something better. If it’s critical for your project, find an expert who can help. Reasons for How could also be managerial or human relations based, perhaps you need to learn how to better manage a team or work with others. But whatever the How reason, there is plenty of answers out there if you know where to look.
What I’m more interested in are the WHY reasons for failing. People wasting their time and money on projects that will never succeed no matter how well they do it. This is surely the more critical of the two reasons for failure because it can strike at any time, no matter how good you are.
A key difference between the two is that there are no ready made answers for getting the Why right, there are only assumptions that need to be tested.
Step 1. Know what type of project you’re working on
From this HOW and WHY logic comes an intuitive next step, are there projects that are more HOW and projects that are more WHY in terms of their failure points? Yes. I believe that all projects fall somewhere on a spectrum between the two. At the HOW end are the logical and routine projects, whilst on the WHY end are projects that are novel, irrational and complex (not necessarily complicated, but complex means there is a unknown number of factors involved).
Routine projects can establish what the correct problem to be solved is at the start. Novel projects will only know if the correct problem has been solved at the end.
Routine (HOW) Projects:
You’ve done it or something very similar to it before and the customer is clear with what they want. How? Because it’s also something routine for them too. It’s a problem based on logic and rational thought with a predictable outcome.
The design of an engineering component is routine, the system within which it is working is known and understood and everything is governed by the laws of physics. Success will be determined by whether it works or not.
Routine does not mean simple, the design of a nuclear power station is anything but simple. However under this definition I would still class it as Routine. The client knows what they want, lots of people have built them before. If at the end of the day (even if hugely overbudget and many years late) you turn it on and it works, then your project has been a success.
Why call them routine? Because these are the logical, pattern based projects that any profession trains for, engineers, doctors, builders etc… We’re all very good by training at dealing with routine problems. But we all struggle far more when we encounter a Novel project…
Novel (WHY) Projects:
Something new, you’re not sure how it will turn out and the customer is also unsure what they want. Novel problems are complex, involve emotion and irrational behaviour. Complex problems involve an unknown number of unknowns, with the relationships between them also complex.
Because people are irrational and not logical creatures (or at least many of them are not), designing anything, no matter how simple, for people is by default a novel problem.
Just to be clear, projects sit on a spectrum, and projects involving many elements may have some that are novel whilst others parts routine, and whilst most work activities might be considered to be routine, changing the status quo will always be novel, especially if the success or failure of the project is dependant on humans and human behaviour.
A fascinating analogy for thinking about routine of novel (complex) problems is that of the mechanic vs the gardener. Sue Goss has completed some really interesting research into this and comparing it to government and how local government is run. It’s well worth investigating further if you’re interested in the subject.
Routine projects can establish what the correct problem to be solved is at the start. Novel projects will only know if the correct problem has been solved at the end.
Step 2. Be aware what the road ahead looks like
Both routine and novel projects share many of the same activities but the order and relationship between them is very different.
Routine projects are linear in nature, establish what the problem to be solved is, design your solution and execute. Very common in a whole host of paid development work. The customer asks for something specific, your deliver it and everyone is happy, often paying upfront or once the problem to be solved is agreed.
Novel projects on the other hand are parallel. The problem finding stage and the problem solving stage happen together, at the same time. You get a hint what the problem might be and as you try to solve it discover more and more what the real problem is. Only finding out at the end when your offering is launched whether you got it right or not.
The biggest issue I see is development teams thinking every project is routine, thinking they can work out what the problem to be solved is at the start. But for many projects this is impossible.
If I start on a film and right away know the structure – where it’s going, the plot – I don’t trust it. I feel like the only reason we’re able to find some of these unique ideas, characters, and story twists is through discovery. And, by definition, ‘discovering’ means you don’t know the answer when you start.
Ed Catmull, PIXAR (Creativity Inc)
“The dogmas of the quiet past are inadequate to the stormy present. The occasion is piled high with difficulty and we must rise with the occasion. As our case is new, we must think anew and act anew. We must disenthrall ourselves, and then we shall save our [project].”
Abraham Lincoln
Highlighting the Key Differences
For a Routine problem you need to establish up front the key deliverables, the key requirements and establish a clear business case before proceeding. Sadly for Novel problems this all comes at the end.
So instead the emphasis is on rapid creativity, rapid prototyping and rapid testing. Exploring the problem space, continuously seeking to understand, validate and confirm customer requirements, wants and demand. Only at the end will you truly know if you got it right, but if you follow a reflective process you’ll always learn from your mistakes.
One key difference often seen in projects is the point at which the customer commits to it. For Routine projects (or projects thought to be Routine) the customer often commits to funding the project right at the start after a brief clarification of the task. For Novel projects it is not uncommon for an income to be generated only at the end after launch… do any customers turn up to buy what we have made?
Reminder: Large projects may have many smaller projects within them, the overall leader may have a Novel task on their hands, steering the whole ship, despite all the sub-projects being Routine to the developers working on them.
“We shall not cease from exploration, and the end of all our exploring will be to arrive where we started and know the place for the first time.”
T.S. Eliot
Step 3. Adjust your priorities accordingly
Being on a path feels better than not, but simply following one doesn’t mean it will take you where you want to go, it doesn’t guarantee success. Does the client / boss / your own brain really know what they’re talking about? Why are we solving this problem at all? Will anyone care? What if we did this, would that be better?
Stop and think, what did I just do? Which reason is it addressing? With two reasons comes two personality types, and whilst we are all on a spectrum between them, or can put on different hats for different situations, the most important bit is to stop and reflect on which hat you’re wearing and is it the right one for this point in time?
Builders follow a path
The boss has an idea, the design brief says this, the client asked for that… We’ve got a plan, a drawing, a design, an activity and we’re going to do it to the best of our ability. Like building a house, we’ve done it many times before and everyone knows what to expect.
The builders are the pure problem solvers, we all love to be them, but unless what you’re doing is routine, to be successful you will need to constantly stop and look around. Is this problem worth solving? Asking that question once at the start is not enough.
The leaders champion Vision.
Explorers make a map
Explorers need to move quickly, do only what is needed to get to the next landmark, the next ridge or the next data point. The explorer is constantly seeking validation that what they’re doing, where they’re going is helpful to the end goal. They might not make things very well, but it’s good enough to test a theory, good enough for now.
These are the problem finders, from their findings the best paths can be made. Builders are essential, but if you’re doing anything novel, you need an explorer (or an exploring mindset) leading the effort.
The leaders champion Purpose.
The 2 golden rules:
Every project is ‘Novel’ until proven otherwise.
Every problem is ‘Fuzzy’ until proven otherwise.
Routine or Novel
An interesting observation when looking at things through the lens of either being Routine or Novel is you can start to see how different stakeholders react to the same project.
Guide
A gift of a situation for the developer, the customer is unsure what they want, but you see the solution clearly. Possibly because you’ve seen this a hundred times before. Quite a common situation in the early days of the internet. Where simple template websites would dazzle and amaze customers who didn’t know what was possible and would pay handsomely for something technically simple. Low risk and potentially very profitable for the developer and although the client might not know it, low risk for them too as they are in a safe pair of hands.
Explore
Projects that are novel for both of you require an exploring attitude, you should be questioning everything and searching for the real need and target of the development. It’s novel for the customer too, so question what they tell you, what they think they need and why they think they need it. Clearly a high risk situation for the client, they are not sure what they really need and it could also be a high risk for the developer too. However I would argue that the perception that “all novel projects are hard” is so heavily weighted in the favour of the developer that they are unlikely to receive any long term negativity if this fails. The client on the other hand is shouldering all the reputation and financial risk.
Build
You know what to do and the customer knows what they want. A loft conversion, a larger version of X, a different colour Y, whatever. Routine and relatively simple things that can be clearly articulated and understood… Draw up a plan and off you go. Projects like this (if they truly are routine!) are low risk for everyone. However I would argue that more projects than many think are in fact not routine at all. As demonstrated by the lists of big product failures each year. Builder categories are low risk but also very competitive and profit potential is limited or clearly defined.
Gamble
It’s routine for them, but you’re lost and somehow you’ve managed to get this opportunity. I’ve called this the gambler because frankly the odds are against you that you’ll pull it off to the satisfaction of the customer. They want something specific and you don’t know what that is. This is typical of developers when starting out. So expect a very high workload as you try to prove you’re up to the task. High risk for both, the developer doesn’t know what they are doing and fears being found out whilst the client believes everything is routine and that success is more or less guaranteed. As a result they are likely to build expectations too high and not manage it carefully enough.
We can’t solve problems by using the same kind of thinking we used when we created them.
Albert Einstein