Evidence-Based Design
Evidence-Based Design (EBD), also referred to as Evidence-Based Software Engineering, is a methodology that incorporates evidence in the decision-making process of software design.
“I believe in evidence. I believe in observation, measurement, and reasoning, confirmed by independent observers. I'll believe anything, no matter how wild and ridiculous, if there is evidence for it.” Isaac Asimov.
At its core, EBD embodies the scientific method: it empowers you to ask the right questions and develop hypotheses, gather quantitative and qualitative data that support or disprove your hypotheses, and measure, share and learn from the outcomes. EBD helps you find product-market fit: EBD will empower you to quantify, understand, and solve customers’ problems. In doing so, you will have a greater chance of creating a successful product. EBD also motivates data-driven teams and helps foster a culture of making decisions based on facts instead of personal preferences or egos. EBD originated in the medical field, popularized in 1984 by Ulrich and colleagues in a paper where they showed the benefits of using rigorous research to link the physical design of hospitals with healthcare outcomes. Today, EBD is used across many disciplines, including architecture, interior design, education, and software.
According to this study of EDB applied to software development, the steps to follow are: ask an answerable question; find the best evidence; critically appraise the evidence; apply the evidence and evaluate performance.
We describe how some of the key processes of Lean Customer Development can drive EBD.
1) Ask an answerable question
While working in a software organization, the clearest evidence you can get to support your hypotheses is customer behaviour. The first step a software team building products should take is to understand who the customer is and what problems the customer has. Try to write down who your target audience is and what problems they face - be as specific as possible. A sample could be:
- Target Audience: enterprise data scientists in the manufacturing industry working on predictive maintenance applications.
- Problem Hypothesis: Preparing data for predictive maintenance applications is the most time consuming and costly task for the audience.
Allow your problem hypothesis to generate answerable questions about the customer’s behaviour What is a customer doing today? How much time do they spend doing the above, and what are the pain points? Do they do anything to work around the problem?
2) Find the best evidence
The current behaviour of customers is a clear signal of what’s really painful and what is a problem people can live with. For example, almost everyone will immediately buy a rat trap if there’s a rat in their house. However, people may take weeks if not months to fix a dent in their car.
Try to reach out and speak to real customers who fall into your target demographic. If you already have customers who use your product, they are a good starting ground for validation. If not, try to find members of your target audience via LinkedIn or GitHub and reach out directly! Make sure to reach out to several people in your target audience - the more data points you can get the better.
In some cases, it is difficult to get direct answers from the customer. A great secondary source of evidence is to analyse what competitors are doing in your product space.
3) Critically appraise the evidence
When gathering evidence, it is very important to be aware of the confirmation bias. Humans have a natural tendency to search, interpret and favour information that confirms our preconceived beliefs or hypothesis. By critically appraising the information one gathers, bias can be avoided.
It is also important to analyse the quality and the rigour of the sources. Many scientists have warned that the pressure of publishing research papers have reduced the overall quality.
4) Apply the evidence
After analysing and understanding evidence of what problems your customer has, the next step is to come up with a solution. If a customer’s problem is that their horse is too slow, there are 100 things that could be viable solutions: you could breed a faster horse, relocate the user to a more central location, or invent the car.
Applying the evidence to your solution can also involve satisfying constraints like meeting customer requirements, technology limitations, or timelines.
5) Evaluate performance
Establish clear and concise KPIs for success before you take on a project based on the evidence. Ask yourself what business or technical metrics would indicate success for the project? Establish target dates for a deliverable and define what is and is not in scope for your deliverable. Prior to your target date, evaluate if you are on track to meet the KPIs you set prior to the project. If not, why?
Make sure to validate your solution with the same audience you spoke to in step 2. After you have a minimally viable product, speak again to your target audience. Ask them what they would do differently given the new solution’s existence. Use the feedback you receive to help iterate and make your solution even better. Even a wireframe or paper mock-up is extremely useful to gather feedback.
