What we know, and mostly don’t know about engineering practices

This is the script for my REES-AAEE-2021 Keynote. The video is here, and the powerpoint slides are available on request if you would like to use them for education purposes.

For a sustainable future, we need large productivity improvements. Engineers are critical contributors, but we need deeper understandings of engineering practices and how education influences them to make the necessary improvements. Without this, education reform arguments are fragile at best.

Here is the script…

Slide 1
This is the shore of Matilda Bay where we are meeting today, for those of you who cannot be with us. This is the land of Wadjuk people of the Noongar nation who gathered at this beautiful place for many tens of thousands of years before us.
Thank you, Sally, Andrew and the REES / AAEE organizing committee for awarding me the privilege to speak to you all.
The title is borrowed from a 2005 paper by Stephen Barlow. While there’s some progress since then, I shall point out critical unanswered questions that still impede most engineering education research and reform efforts.
I shall argue that large improvements in engineering performances are possible: we will need them to sustain human life as we know it on this planet. That’s why we need more understanding on today’s engineering practices, so we can lay the education foundations for the necessary performance improvements.
My evidence comes from several data sources, including over 300 interviews and a dozen field studies of engineers at work in several countries, in different settings over 20 years, representing all main disciplines.
There is much to be proud of. Engineers rarely encounter difficulties with engineering science in their jobs. Nearly all seem amazingly satisfied with their career choices.
Yet there are still many recurring engineering failures with enormous social and economic costs: the root causes can be traced back to education weaknesses.


Slide 2
I will argue that improving productivity globally is critical for our future. Without this we will not achieve the sustainable development goals, nor avoid catastrophic global warming, nor eliminate poverty, nor will we have political stability.
Productivity is an unfamiliar topic for many engineers. I shall explain the relevance and why engineering performances are so influential.
We will need large improvements in engineering performances in order to achieve the productivity improvements we need.
For that we will need significantly improved understandings of engineering practices and the influences of education.
Along the way we will need answers for several critical questions for us as engineering education researchers.
Finally, because research takes time, I have some short-term suggestions for you to try, that might shift the workplace performance dial a little in the right direction.


Slide 3
Productivity is critical because it affects all aspects of human life on this planet. Take safe drinking water as an example.


Slide 4
Here in Perth, safe drinking water costs around three US dollars per thousand litres, delivered through the kitchen and bathroom taps of your home.


Slide 5
Right across South Asia, women and children have to carry water like this every day.
The cost, including the value of unpaid labour (a real economic penalty), is around USD 35 – 50 per thousand litres. Up to 15 times the cost here in Perth, the driest city in the driest continent.
This engineering failure to deliver affordable safe drinking water for people in homes and workplaces affects around 3 billion people worldwide. Hundreds of millions of children are stunted because of being forced to drink contaminated drinking water through sheer necessity. That’s an almost unimaginable burden of care on future generations. Read about it!


Slide 6
Safe drinking water is still delivered like this in so many countries, even today.
Each bottle costs around USD 2 delivered and contains just 20 litres. That’s USD 70-130 per thousand litres, 30 times the cost in Perth.
While people in low income countries face such extraordinarily high costs, how can we eliminate poverty and foster social development?
These cost differences reflect productivity differences which in turn reflect differences in engineering performances.


Slide 7
This chart shows productivity improvement in advanced countries like the USA, Australia, Europe and others.


Slide 8
This chart shows productivity improvement in the emerging world. South Korea is at the top for comparison.
Countries with the largest populations on the planet average around five times less productivity than wealthy countries. That means their people face much higher costs for goods of similar quality and durability, on average, than us in wealthy countries. This ratio has not changed in over 60 years! Hence the difficulties we see in poverty elimination efforts and humanitarian aid programs.


Slide 9
Reducing greenhouse emissions is all about improving productivity. Improved energy efficiency, providing similar performance with less energy use, will provide about half of all the greenhouse emission reductions we need.


Slide 10
Unfortunately the productivity improvements that would show we are on the path to a more sustainable future are not evident, yet.
Here’s a chart that shows productivity changes in the US manufacturing sector since the 1980s. Notice that there has been negligible improvement since around 2005.
Similar figures are evident in productivity data from nearly all other countries. Rich and poor. Economists debate the underlying causes of this “productivity puzzle”. Many hope that digitisation will bring productivity improvements. I doubt that. To me there is an overwhelming coincidence in these figures…
The introduction of the mobile smartphone in 2005.
I think there is a causal link here: you’re welcome to ask for my reasoning after the talk.


Slide 11
One of the consequences of low productivity growth, when expectations of improved living standards fail to eventuate, is that demagogues like Donald Trump get elected offering slogans instead of solutions.


Slide 12
Now you understand why productivity is so important, I need to explain why engineers are so influential.


Slide 13
The link between engineering performances and productivity was well understood in the 1950s as we can see from this quote from the influential ASEE Grinter report that transformed engineering education from the 1960s onwards.
“The obligations of an engineer as a servant of society involve the continual maintenance and improvement of man’s material environment, within economic bounds, and the substitution of labor-saving devices for human effort”


Slide 14
Today, this link seems to have been forgotten. Yet, when we look at what engineers do, reduced to the simplest possible terms, this is how I describe it:
“Engineers are people with specialised technical knowledge who conceive, deliver, operate and sustain artificial objects, systems and processes. These enable people to do more with less… Less effort, time, materials, energy, uncertainty, health risk and environmental disturbances.”


Slide 15
In other words engineers enable other people to be more productive. You can see that from the dawn of civilisation: take irrigation systems as an example.
And that’s what I still do in my work as an engineer since I retired from being a teaching and research academic a few years ago.


Slide 16
My work should enable everyone to have affordable cooling with far less energy consumption and climate impact than traditional air-conditioning that’s been around for a century or so.
Air cooling in hot climates enables enormous productivity, health and welfare improvements. You only need to look at the history of the USA, Singapore, Japan, South Korea even China to see that.


Slide 17
And here’s my little invention: a tiny fridge with a fan inside that blows cool air, cooling people not buildings. We can achieve comfort this way with far less energy consumption.


Slide 18
The next part of my argument is that there is plenty of space to improve engineering performances.


Slide 19
Let’s look at industry data on capital expansion projects, over $1 billion. An arbitrary criterion for success that the investors should get at least 50% of their promised investment return. That’s a pretty low bar and hardly satisfactory.
The data tells us that only two in every six of these large engineering projects succeeds. One in six are complete write-offs: the investors lose all their money.
Even in well organised engineering enterprises, about 50% of operating turnover is lost from operations and maintenance mistakes.
These figures are appalling. However, because they are so bad, we should be able to get significant improvements relatively easily. That’s the only consolation.
If you want an explanation why engineers have such a rotten reputation amongst investors and governments, you need look no further.


Slide 20
Here’s an example from Western Australia, about 500 km east of Perth.
This nickel refinery cost US$2.6 billion to build and was sold for $250 million in 2009. The buyers thought they had a bargain. They spent $400 million on upgrades and then mothballed it a few years later.
$2.9 billion in value destroyed. And this is one of many, many, many projects worldwide with similar outcomes.
You won’t hear much about these projects because most owners don’t want their shareholders to know.


Slide 21
Collaboration weaknesses provide explanations for most project failures and also most major industrial accidents.
Weaknesses include divergent notions of project objectives: People in different parts of the enterprise have quite different understandings about the project aims.
Weaknesses also include lack of planning and detailed documentation, inappropriate specifications, inadequate checking and testing and inadequate design reviews.


Slide 22
So, what do we know about engineering practices?


Slide 23
There is plenty of evidence now that engineering is predominantly inter-dependent work. (Thank you to Kacey Beddoes for giving me that term). It is socio-technical work in which the social and technical aspects are inseparable. It starts and ends with collaboration.
The largest single component of an engineer’s day involves social interactions with other people to enact collaboration performances. Meetings, emails, telephone conversations, text messages, document exchanges, and so on. A good proportion of the time is spent finding information.
When you ask engineers to assess how much time they spend on solitary technical work like design, calculations, problem solving and so on, the kinds of stuff we teach in universities, most disciplines report 2 to 4%. Some disciplines that involve extensive analysis, simulation or design report up to 20%.
I will try and explain why collaboration is such a large part of engineering, and why social interactions are so dominant, even at the start of an engineer’s career in jobs with predominantly technical focus.


Slide 24
I use this diagram to illustrate the components of engineering practice that came out of the studies that I performed with my students.
Engineering practice rests on a platform of technical and financial foresight: being able to make predictions and plans to enable engineering activities. It involves extensive technical collaboration performances on which results depend.
The three main results I symbolise by 3 cups of tea: my wife is addicted to tea.
1) Client satisfaction triggers payment for the work done: that’s an essential economic outcome for engineers.
2) Intrinsic achievement satisfaction, when you tell your grandchildren “See that funny little yellow thing sticking out of the side? I designed that!”
3) Success builds your reputation as an engineer which in turn leads to further work.
The collaboration performances rely on technical knowledge and understanding permeating everything. I symbolise this idea with these blue traces emanating from the right-hand corner: the technical core of engineering which stems from the education that we all strive to provide.
Yet there are two other critical supports for all engineering performances. On the left-hand side we see perception performances. And in the background what I call tacit ingenuity: knowhow that we are normally unaware of, and which enables us to create solutions.
Colouring at the base of the cones denotes the relative attention that we give these aspects in undergraduate and school education. There is considerable learning asymmetry: there is very little focus on perception skills and tacit ingenuity.


Slide 25
The slab of technical collaboration performances expands into several different sociotechnical performances which I described in my book a few years back:

Discovery learning

Informal teaching

Technical coordination

Uncertainty management

Multi-party negotiations


Slide 26
Here is my representation of the technical knowledge that graduates emerge with from engineering schools. I represent knowledge with clouds for several reasons. It’s hard to grab one: it’s hard to see how deep you are going into one, how much further you need to go, and once immersed in the cloud, it is very hard to see the world outside. I hope this analogy is meaningful for you.


Slide 27
Yet, this knowledge is only a tiny part of the knowledge that engineers rely on.


Slide 28
Here are around 35 types of technical knowledge that emerged from our research and the links between them.


Slide 29
There are also many types of organisational and business knowledge that are equally critical for engineers.


Slide 30
How can an engineering graduate learn all this when almost none was even mentioned in school?


Slide 31
Here we see her: she can cover the knowledge she learned at school.


Slide 32
Others people have specialised and cover some of the other types of knowledge.


Slide 33
Yet more people cover the other areas of knowledge she will need.


Slide 34
She collaborates with all these people to access all the other knowledge she needs. She can’t learn it all from them, not even in 10 lifetimes. Instead, she persuades them to contribute skilled performances using the knowledge they carry in their heads: a process that I have refer to as technical coordination.
This informal collaboration (a kind of informal leadership) depends on relationships: some are strong, illustrated with thick lines. Others are relatively tenuous, denoted with thin lines.


Slide 35
One of my students suggested this analogy: all these people have relationships represented by pipes between them with valves.


Slide 36
(Enlargement)
Each person can open or close the valve governing the strength of the relationship between them.
Engineering knowledge, therefore, is a multi-layered social network, a different layer for every engineering product, process or system. Traversing it takes time and social skills.
Now you can begin to appreciate the complexity of the technical collaboration on which engineering performances depend.


Slide 37
Remember the limited knowledge subset at graduation.


Slide 38
It’s such a small part…
… the narrow blue band you can see here


Slide 39
Now let’s think about the potential of education to influence workplace performances.
Many studies have shown that university performances do not seem to explain adult achievement very well.
If we focus on improving learning in the traditional technical aspects of engineering, I see very limited potential for improved engineering performances.
We could do far more by focusing on collaboration.


Slide 40
Technical engineering learning is such a small part.
Engineering performances are overwhelmingly influenced by collaboration performances. There’s far more potential to influence outcomes by focusing here. But there are many difficulties. Not the least is a curriculum which is already overcrowded.


Slide 41
Another difficulty:
So far, to my knowledge, no one has come up with evidence that contradicts this assertion I offered a couple of years ago…
Exams and written assessment methods cause students to privilege and value independent work and written communication.
To focus on collaboration, we need to balance the powerful hidden messages coming from assessment methods that contradict attempts to get students to embrace collaboration and oral communication skills.


Slide 42
Therefore, engineering education research needs to move into new spaces, particularly the early career stage for young engineers.
My books provide some starting points, but I have to admit that they leave so many questions unanswered.


Slide 43
Culture, systems and structures complicate this issue. There are many influences on engineering performances: general intelligence and learning ability, learned abilities such as emotional intelligence and languages, capabilities and expectations shaped by higher education; the culture of the host society; the culture, systems and structure of the organisation; and the interplay between structures and agency. We are only just beginning to even consider these issues.


Slide 44
To illustrate the concept of structure and agency, think about this street market. Markets like this have existed for millenia. But it takes a lot of experience, social skills, knowledge and determination to build up a loyal customer base enabling someone to earn a reliable income in markets like this.


Slide 45
In contrast, here’s a modern supermarket. The arrangement of shelves, price tags, bar codes, logistics systems, cash registers, product packaging (all incorporating heaps of embodied knowledge)… All this enables young people with little or no experience to make a decent living in an industry with extremely tight margins.
The structure enables people to be productive without much skill development. At the same time it limits their autonomy.


Slide 46
Perception skills and tacit ingenuity, two of the three pillars for engineering practice, are hardly developed at all relative to engineering science. So how is this apparent disaster averted?
Just like the supermarket, it is the structures in established engineering enterprises: formal procedures and ways of working, standards, codes, regulatory frameworks. Engineers with patchy skills (even patchy science and maths) can fit into these structures and work productively. They necessarily limit an individual engineer’s autonomy (or agency).
The lesson here is that individual performance depends as much or more on workplace culture, systems and structures as individual capabilities.
Understanding education influences will not be easy.


Slide 47
We are left with so many unanswered questions.
As Celia has pointed out in her talk, we’re not even sure how to measure competencies.
All the papers I have seen derive competencies from engineers’ or employers’ opinions. However, in our research, we found that engineers were unaware of many aspects of the work they perform. I am still to be convinced that engineers can provide reliable competency descriptions.
How well do competencies predict workplace performances? A question waiting for answers. I have not seen any studies on this.
Now,
How can we assess education interventions without understanding their effects on engineering workplace performance? I’ve seen only a few studies and the results were not encouraging.
Finally, I ask you, how can we argue for curriculum changes without answers to these questions?
My book, “Learning Engineering Practice” comes with an online appendix, a professional engineering capability framework that I developed with Engineers Australia. It sub-divides engineering performance into a couple of hundred components, each of which might be reasonably easy to teach and perhaps measure. It’s just a suggestion: you’re welcome to use it or adapt it.


Slide 48
All this is going to take a long time to sort out. So here are some short-term possibilities that you might like to try.


Slide 49
Why is global productivity not growing fast enough? In my analysis, a big part of the answer is that we stopped teaching engineers that it is their job around 50 years ago. Today we teach them that their job is to solve technical problems.

Sure, engineers solve technical problems from time to time. But so do cooks, housekeepers, business executives, lawyers, accountants. People solve problems all the time, like how to find a convenient parking spot every day they drive to work.

In fact, in our research, we found that good engineers are people who know how to avoid problems.

I argue that we a definition of engineering that shows how engineers improve productivity: one that explains the social purpose of engineering. If we fail to teach engineers that it is their job to improve human productivity, we should not be surprised if productivity does not improve.

<–>
When I started teaching listening and notetaking skills in a capstone design project course, the students asked me why they did not get help to learn that in first year. They found it really helpful and research has demonstrated the learning benefits.

<–>
Many years ago I learned that we could build students visual perception skills by getting them to learn freehand sketching. Sheryl Sorby and others have developed these ideas extensively.
<–>

From limited evidence, it is apparent that engineers are weak in critical thinking skills. Recently I had to accredit an engineering course with around 25% of the curriculum devoted to humanities and social sciences. I expected the graduates to have an excellent appreciation of critical thinking and outstanding communication skills.
<–>

It might seem surprising, that both alumni and employers still complained about communication skills. I believe they conflated communication with collaboration skills. The students and alumni that we met on the accreditation visit were extraordinarily articulate. But not necessarily good collaborators. And, furthermore, they could not distinguish critical thinking from design thinking: they were unable to explain what critical thinking really is.
<–>

Having seen it used at this university for many years, I think that the routine recording of lectures is one of the most destructive education innovations introduced in my entire career. It promotes disengagement, absence from the campus, and weakens the incentive for students to learn social skills which largely determine career success.
The design capstone course that I taught for the last few years of my career relied on classroom attendance and participation. Andrew Guzzomi will tell you how he managed to do that through the pandemic.
<–>

We need to reward students for interdependent performances. Role-play simulations are an excellent way to do that. I’m happy to elaborate more.
<–>

I argued many years ago that engineering students should learn to teach. Students learn much more from each other than they do from us. Why not build on that and, in doing so, reward them for interdependent performances? Grade them on the results achieved by the students they teach rather than their ability to recite pedagogy theories.


Slide 50
That’s it.
There’s some recent literature for your reading list. And some of my photos of Perth and other places in Western Australia to watch while you ask me some questions.

Changing notions of comfort

I am so thankful I don’t have to work all the time in an air-conditioned office building. Especially since Covid-19, our entire Close Comfort team works part of the time at home. We’re happier and feel healthier too.
Of course, I have a Close Comfort personal air conditioner with me. Our team members each have at least one at home as well.
Lee Kuan Yew, honoured as Singapore’s founding father, loved to tell everyone how air conditioning enabled today’s Singapore by providing a comfortable working and sleeping environment. However, there’s a dark side that comes with 20th-century air conditioning systems.
It is well established that people who live most of the time in constant temperature air-conditioned buildings lose their natural thermal acclimatization. As a result, they only feel comfortable at about 23 °C.
Recently I hailed a Singapore cab and climbed into the shiny black refrigerator on wheels, feeling so glad I remembered to bring a cardigan tied around my shoulders. The driver exclaimed, “Ah, it’s so hot today, la!”
“What’s the temperature?” I asked.
“33, it’s really hot, la”.
“But, yesterday it was 32”.
“Yeah, 33, it’s so hot today, la!”

Continue reading

Lost in Urdu translation?

Today with the internet and web sites like Freelancer.com, you would think it is possible to find native speakers of any language who can readily do translation for you and prepare documents like instruction manuals.  And with computer packages like InDesign, you would think that the job of setting up a professionally printed instruction booklet again would be trivially easy.  Well, maybe in some languages, but not in Urdu.  This has been a fascinating learning experience, and frustrating too at times.

I never imagined that translating our air conditioner user manual and getting it designed ready for printing would have been so difficult.  This manual is critical for us: few people will read it, but the few who do will pass on that knowhow to lots of other people.  It’s really important that they get it right.

Continue reading

Lahore Book Launch

“The Making of an Expert Engineer” was officially launched in Lahore at the Avari Hotel on March 3rd before a gathering of 120 engineers, engineering faculty, aspiring engineers, and friends.  Prof. Fazal Ahmad Khalid, Vice Chancellor of the Lahore University of Engineering and Technology (UET) presided at the launch.  The launch was sponsored by the author’s company Close Comfort.

DEEJ2227

James Trevelyan demonstrating the Close Comfort bed tent with air conditioner.

Audio Recording:

Selection of photographs taken at the event

DEEJ2420

James Trevelyan speaking about the book

DEEJ2603

Prof Fazal Ahmad Khalid

Continue reading

Pakistan Launch: Islamabad

“The Making of an Expert Engineer” was officially launched in Islamabad at the Serena Hotel on January 7th before a gathering of 120 engineers, engineering faculty, aspiring engineers, and friends.  The Hon. Ms. Marvi Memon, Minister Chair of Benezir Bhutto Income Support Fund spoke about the potential impact of the research on the poorest 5.8 million people in Pakistan served by the fund. Lieutenant General (R) Syed Shujaat Hussein, former rector of National University of Science and Technology presided at the launch.Close-Comfort-FB-Logo-151207

The event was sponsored by Close Comfort Air Conditioning

MH4_0499-Edit

James Trevelyan speaking about the book – transcript of speech appears below.

Continue reading

A Bright Energy Future for Pakistan

Despite current on-going energy shortages and load shedding, Pakistan has energy wealth that could be unlocked just by thinking differently about electricity distribution.

Electricity distribution

Electricity distribution systems are large engineering enterprises (photo from http://en.wikipedia.org/wiki/Electric_power_transmission)

Electricity supply is capital intensive engineering. Pakistan built the existing electricity supply network with the help of large loans on favourable terms from the World Bank and other international institutions.

In addition, Pakistan has benefited from the generosity of Saudi Arabia in providing low-cost fuel.

Pakistan has reaped the benefits of large hydroelectric generating plants at Mangla, Tarbela and other dams: they generate electricity with no ongoing fuel costs.

As fuel and capital borrowing costs rose for Pakistan in the last 20 years, and the proportion of cheap hydro power reduced, Pakistan governments shielded people from the real cost of electricity generation with generous subsidies but these cannot continue.

Another factor that frustrates efforts to find energy solutions is the high cost of engineering in Pakistan. Through research we have identified many factors that Pakistan engineers struggle to overcome, such as the deep social divides that inhibit effective collaboration and knowledge sharing between engineers, investors and labour. Given the same requirements for product availability and service quality, the cost is almost invariably higher in Pakistan than in industrialised economies like Europe and the USA. Just as an example, when indirect costs are taken into account, the cost of safe drinking water ranges from US$50 to $150 per tonne in Pakistan while the cost in Australia, the driest continent, is US$3 per tonne.

(This is an updated and extended version of an article published in The News, Pakistan, 31st May 2013)

Continue reading

Challenges for Pakistan Engineers

A Pakistan university Vice Chancellor told me how, when he first took up his position, he challenged his engineering faculty.

“Listen, he said, you and other engineering schools in Pakistan have graduated tens of thousands of electrical engineers, yet, the more you graduate, the worse electricity load shedding becomes.”

“Sir, they replied, that is a political problem, it’s nothing to do with engineering! The politicians have accumulated a huge circular debt, which is not real debt, just an accounting aberration to cover the fact that rich people don’t pay for electricity.”

The Vice Chancellor smiled. “Please remember, he said, electricity and water utilities are staffed and run by engineers. Furthermore, the debt is real debt: Pakistan State Oil now has to pay cash in advance of delivery because it ran up too much unpaid debt with suppliers. As long as people can use electricity without paying enough to cover the cost of fuel to run generators and maintaining and extending all the transformers and cables, the problem will get worse. So whether you like it or not, as far as Pakistan is concerned, it is an engineering problem. That means it’s your problem too!”

Pakistan’s politicians and business community have a low opinion of Pakistan engineers: it is not just load shedding and poor water service quality. Pakistan is a high cost operating environment, and Pakistan engineers (with a few notable exceptions) have a poor record for delivering on promises: on-time, with good quality, high safety standards, and within financial constraints. In short, Pakistan is an unattractive destination for capital investment because engineers (among others) don’t deliver what they promise.

That’s the bad news.

There’s good news, too….. well sort of. Continue reading

A moratorium on new coal mines: a hypocritical Australian gesture

Recently there has been an Australian move to propose a moratorium on new coal mines.

For several years I have researched energy supplies on the ground in India and Pakistan.  I have also researched how engineers respond to the challenges of energy and water supplies there, and also in Australia.

I strongly disagree with this moratorium proposal.

Why?

First, it will be seen as hypocritical and selfish in countries like India and Pakistan because we Australians, more than many countries, have grown rich and prosperous by burning vast quantities of coal in the past and continue to do so today.

Continue reading

Back from Pakistan, UAE, Iran and New Zealand

Some of you may have wondered why there has been a little gap in my blog posts.  I have been pre-occupied with visits to several countries.

My other major project, Close Comfort has developed very quickly with keen anticipation particularly in Pakistan where electricity supplies are subject to frequent interruptions due to load shedding.  Pakistan’s electricity grid is struggling to keep up with demand for air conditioning, and I hope to be able to offer a sustainable solution, as explained in Chapter 13 of the book. Continue reading