Skip to main content

The danger in relying on evidence for decisions

We often make decisions based on the evidence in front of us. That may or may not be a good thing. What about the evidence that we don’t see? Freek Vermeulen gives an example of drawing the wrong conclusion from the evidence in front of you.

During the Second World War, engineers looked at the damage caused by enemy fire on returned aircraft, and increased protection in those areas of the plane most hit. That was completely the wrong thing to do. Damaged aircraft were able to return to base if they were hit in non-critical areas. Aircraft hit in critical areas didn’t return, and it was those unknown critical areas that needed reinforcing. The evidence in front of the engineers not only was insufficient for a right decision but also led them to make a wasteful decision.

What’s the vital evidence that you don’t have?

First posted March 26th, 2009.  One thoughtful interpretation I've heard since then is the risk of asking only your loyal customers what needs improving in your product (like the plane that came home despite damage).  You're unlikely to hear from satisfied customers why others stopped buying your product, or why some people never bought it at all.  So you spend time, money and effort improving value for your loyal customers.  However, some well-meant customer suggestions don't lead to increased price or volume, and you may have missed the opportunity for changes with much greater potential for retaining or attracting other customers.