In this series of posts, I am going to show how engineering underpins the world economy more than we think, and how we can improve our engineering performances by changing the way we think about engineering.  The last post was bad news for us engineers.  The good news is that we can all gain by improving our engineering performances.  However, to turn our performance around, we first need to understand what’s not working.

Companies like IPA Global have answers based on statistical analysis, and have provided these answers for several years.  They will tell you which factors are statistically correlated with successful projects, as Ed Merrow has written in his book.  However, even the projects they interact with are getting worse, and there are many more projects that they don’t assess, such as government engineering projects.  Political constraints with these projects usually rule out closing down a failing project: unemployment is often a bigger issue than a failed project for government sponsors.

Clearly there are other factors at work here.  The fundamental difficulty with statistical correlations is that they cannot provide causes.  Try this example.  My hair grows every day and Halley’s comet is moving further from the sun every day.  But that does not mean that my hair will get shorter when Halley’s comet comes back towards the sun.  Statistical correlations can tell you which factors are correlated with project outcomes, but these associations cannot tell us much about the causes of project failures or how to make improvements.

We have to turn to different kinds of research to find the underlying causes for engineering project failures.  The qualitative ethnographic research we do with engineers can help identify potential causes that statistical correlations miss.

We can be confident that there is no single reason why so many engineering projects fail to achieve expected results so regularly.  However, interviews with over 350 engineers have helped to identify factors that almost certainly contribute.

Weakness in systematic document checking is one such contributing factor.

We discovered this when a company asked us to study their engineers to see if we could find out why their projects consistently ran late and over budget.  The company was based in Australia and had built a nation-wide EPCM (engineer procure construct manage) business building mineral processing plants.

At the time, we were also interested in studying design checking: how engineers go about checking for mistakes in designs.  We accidentally discovered that engineers were delegating the checking work to the most junior staff because they were “too busy” to get the irksome checking work done.  Once the checks were completed, the engineers wrote their initials on the documents to indicate the checking had been done.  The junior checkers were the least experienced people, often just students working part-time to earn some pocket money.  The engineers, of course, would never admit that they had delegating the document checking when they signed their initials to certify they had done the checking themselves.

Our challenge, therefore, was how to gather systematic and reliable evidence on what was actually happening.

There was a strong likelihood that questions such as “tell us how you do your checking work” would have been answered with normative generalisations. Like this: “Ah well, when you’re checking documents, it is really important to check the assumptions.”

Therefore, instead, we asked engineers to tell us about the mistakes they had discovered in documents they had checked recently.  We were surprised by two consistent themes that emerged when we listened to what they told us.  First, the only mistakes they found were associated with work performed by people in other companies.  No one mentioned mistakes in work by performed by company staff.  Second, nearly all the mistakes they described were minor ones such a grammatical and spelling errors.

These observations strengthened earlier evidence that the engineers were not performing the checking work themselves, and also suggested a culture in which errors were not openly discussed within the company.  This was to be confirmed later.

At the same time, we learned that the reason why their projects were running consistently late was that mistakes were being found late in the design process, so a large number of documents had to be fixed to correct mistakes.  For example, several engineers commented that the details of externally supplied equipment for the processing plants turned out to be different from the initial assumptions.  The initial assumptions came from data on equipment used in previous projects but the equipment vendors had subsequently changed their designs.  However, the company engineers had not asked the equipment vendors for accurate up-to-date CAD files early in the design process so the design changes only became apparent late in the project, or when the equipment arrived on site and did not fit.

When we asked how engineers felt about being asked to perform checking work, we found something else that was interesting.  Engineers thought checking was unproductive.  Here’s a typical reply: “It’s not like you’re doing something productive or creative, like designing or doing modelling calculations. It’s taking time and money and stops you from getting on and making progress elsewhere.”

Some engineers knew it is valuable to find mistakes as early as possible in the design process: “a mistake that costs $1000 to fix at the design stage could cost $1 million or even $10 million to fix on site.”

Gradually a picture emerged that helped us understand why the checking work was delegated.  Most of the engineers were under constant pressure to try and avoid further schedule delays, and thought that checking work delayed other work needed to get the project completed.  They thought checking was unproductive and boring, and was not creating value.

When we provided the final report to the company, they declined to discuss it.  The company employed external auditors to check that every aspect of their design quality assurance process complied with the company procedures, and they were proud of their “zero non-conformance” record.  Possibly they were reluctant to admit that the reality of design checking was quite different from the reports provided by the quality assurance auditors.

Shortly afterwards and quite coincidentally, we were asked to perform a similar study by a different company, an international player in the energy resources industry.  They were experiencing similar issues and our research demonstrated a similar result.  Even though they had a strict quality assurance system with rigorous external auditing revealing zero non-conformance, the results were similar and the company was equally reluctant to discuss the results of our study.

Subsequently engineers have told us about similar issues in so many different companies.  Why is this pattern of behaviour by engineers so consistent?

Here’s our best explanation so far.

We engineers are largely autonomous: mostly we decide for ourselves what we do each day.  Every day we make countless instinctive decisions, like which email to open first, or which people to chase on the phone.  All these decisions imply relative priorities: there is never enough time to attend to everything.  In other words, there is an implied hierarchy of value: certain activities are more valuable than others.

Psychologists studying education and motivation have provided some useful ideas.  Value expectancy theory explains how people choose activities that they think they can do well (expectancy) and will contribute something useful (value).  Choices reflect often unconscious personal understandings of proficiency (or self-efficacy) and perceived value.  Value is multi-dimensional: for example, value can be instrumental (what we get from performing a task) or intrinsic (the fun or enjoyment value) or a combination.

What’s important here is that engineers’ choices on checking work reflect seem to reflect unconscious ideas on what contributes value.

We learn from this that changing value perceptions may be a more constructive way to address the weaknesses in design checking.

We will see this again in some coming posts that continue on the theme, examining reasons why large projects fail.

If you have an interesting story to share with me, please use the form below to provide some details.

For further reading:

http://staffhome.ecm.uwa.edu.au/~00006605/eng-work/publications/Thesis-Mehravari.pdf