This is the script for my REES-AAEE-2021 Keynote. The video is here, and the powerpoint slides are available on request if you would like to use them for education purposes.

For a sustainable future, we need large productivity improvements. Engineers are critical contributors, but we need deeper understandings of engineering practices and how education influences them to make the necessary improvements. Without this, education reform arguments are fragile at best.

Slide 1
This is the shore of Matilda Bay where we are meeting today, for those of you who cannot be with us. This is the land of Wadjuk people of the Noongar nation who gathered at this beautiful place for many tens of thousands of years before us.
Thank you, Sally, Andrew and the REES / AAEE organizing committee for awarding me the privilege to speak to you all.
The title is borrowed from a 2005 paper by Stephen Barlow. While there’s some progress since then, I shall point out critical unanswered questions that still impede most engineering education research and reform efforts.
I shall argue that large improvements in engineering performances are possible: we will need them to sustain human life as we know it on this planet. That’s why we need more understanding on today’s engineering practices, so we can lay the education foundations for the necessary performance improvements.
My evidence comes from several data sources, including over 300 interviews and a dozen field studies of engineers at work in several countries, in different settings over 20 years, representing all main disciplines.
There is much to be proud of. Engineers rarely encounter difficulties with engineering science in their jobs. Nearly all seem amazingly satisfied with their career choices.
Yet there are still many recurring engineering failures with enormous social and economic costs: the root causes can be traced back to education weaknesses.

Slide 2
I will argue that improving productivity globally is critical for our future. Without this we will not achieve the sustainable development goals, nor avoid catastrophic global warming, nor eliminate poverty, nor will we have political stability.
Productivity is an unfamiliar topic for many engineers. I shall explain the relevance and why engineering performances are so influential.
We will need large improvements in engineering performances in order to achieve the productivity improvements we need.
For that we will need significantly improved understandings of engineering practices and the influences of education.
Along the way we will need answers for several critical questions for us as engineering education researchers.
Finally, because research takes time, I have some short-term suggestions for you to try, that might shift the workplace performance dial a little in the right direction.

Slide 3
Productivity is critical because it affects all aspects of human life on this planet. Take safe drinking water as an example.

Slide 4
Here in Perth, safe drinking water costs around three US dollars per thousand litres, delivered through the kitchen and bathroom taps of your home.

Slide 5
Right across South Asia, women and children have to carry water like this every day.
The cost, including the value of unpaid labour (a real economic penalty), is around USD 35 – 50 per thousand litres. Up to 15 times the cost here in Perth, the driest city in the driest continent.
This engineering failure to deliver affordable safe drinking water for people in homes and workplaces affects around 3 billion people worldwide. Hundreds of millions of children are stunted because of being forced to drink contaminated drinking water through sheer necessity. That’s an almost unimaginable burden of care on future generations. Read about it!

Slide 6
Safe drinking water is still delivered like this in so many countries, even today.
Each bottle costs around USD 2 delivered and contains just 20 litres. That’s USD 70-130 per thousand litres, 30 times the cost in Perth.
While people in low income countries face such extraordinarily high costs, how can we eliminate poverty and foster social development?
These cost differences reflect productivity differences which in turn reflect differences in engineering performances.

Slide 7
This chart shows productivity improvement in advanced countries like the USA, Australia, Europe and others.

Slide 8
This chart shows productivity improvement in the emerging world. South Korea is at the top for comparison.
Countries with the largest populations on the planet average around five times less productivity than wealthy countries. That means their people face much higher costs for goods of similar quality and durability, on average, than us in wealthy countries. This ratio has not changed in over 60 years! Hence the difficulties we see in poverty elimination efforts and humanitarian aid programs.

Slide 9
Reducing greenhouse emissions is all about improving productivity. Improved energy efficiency, providing similar performance with less energy use, will provide about half of all the greenhouse emission reductions we need.

Slide 10
Unfortunately the productivity improvements that would show we are on the path to a more sustainable future are not evident, yet.
Here’s a chart that shows productivity changes in the US manufacturing sector since the 1980s. Notice that there has been negligible improvement since around 2005.
Similar figures are evident in productivity data from nearly all other countries. Rich and poor. Economists debate the underlying causes of this “productivity puzzle”. Many hope that digitisation will bring productivity improvements. I doubt that. To me there is an overwhelming coincidence in these figures…
The introduction of the mobile smartphone in 2005.
I think there is a causal link here: you’re welcome to ask for my reasoning after the talk.

Slide 11
One of the consequences of low productivity growth, when expectations of improved living standards fail to eventuate, is that demagogues like Donald Trump get elected offering slogans instead of solutions.

Slide 12
Now you understand why productivity is so important, I need to explain why engineers are so influential.

Slide 13
The link between engineering performances and productivity was well understood in the 1950s as we can see from this quote from the influential ASEE Grinter report that transformed engineering education from the 1960s onwards.
“The obligations of an engineer as a servant of society involve the continual maintenance and improvement of man’s material environment, within economic bounds, and the substitution of labor-saving devices for human effort”

Slide 14
Today, this link seems to have been forgotten. Yet, when we look at what engineers do, reduced to the simplest possible terms, this is how I describe it:
“Engineers are people with specialised technical knowledge who conceive, deliver, operate and sustain artificial objects, systems and processes. These enable people to do more with less… Less effort, time, materials, energy, uncertainty, health risk and environmental disturbances.”

Slide 15
In other words engineers enable other people to be more productive. You can see that from the dawn of civilisation: take irrigation systems as an example.
And that’s what I still do in my work as an engineer since I retired from being a teaching and research academic a few years ago.

Slide 16
My work should enable everyone to have affordable cooling with far less energy consumption and climate impact than traditional air-conditioning that’s been around for a century or so.
Air cooling in hot climates enables enormous productivity, health and welfare improvements. You only need to look at the history of the USA, Singapore, Japan, South Korea even China to see that.

Slide 17
And here’s my little invention: a tiny fridge with a fan inside that blows cool air, cooling people not buildings. We can achieve comfort this way with far less energy consumption.

Slide 18
The next part of my argument is that there is plenty of space to improve engineering performances.

Slide 19
Let’s look at industry data on capital expansion projects, over $1 billion. An arbitrary criterion for success that the investors should get at least 50% of their promised investment return. That’s a pretty low bar and hardly satisfactory.
The data tells us that only two in every six of these large engineering projects succeeds. One in six are complete write-offs: the investors lose all their money.
Even in well organised engineering enterprises, about 50% of operating turnover is lost from operations and maintenance mistakes.
These figures are appalling. However, because they are so bad, we should be able to get significant improvements relatively easily. That’s the only consolation.
If you want an explanation why engineers have such a rotten reputation amongst investors and governments, you need look no further.

Slide 20
Here’s an example from Western Australia, about 500 km east of Perth.
This nickel refinery cost US$2.6 billion to build and was sold for $250 million in 2009. The buyers thought they had a bargain. They spent $400 million on upgrades and then mothballed it a few years later.
$2.9 billion in value destroyed. And this is one of many, many, many projects worldwide with similar outcomes.
You won’t hear much about these projects because most owners don’t want their shareholders to know.

Slide 21
Collaboration weaknesses provide explanations for most project failures and also most major industrial accidents.
Weaknesses include divergent notions of project objectives: People in different parts of the enterprise have quite different understandings about the project aims.
Weaknesses also include lack of planning and detailed documentation, inappropriate specifications, inadequate checking and testing and inadequate design reviews.

Slide 22
So, what do we know about engineering practices?

Slide 23
There is plenty of evidence now that engineering is predominantly inter-dependent work. (Thank you to Kacey Beddoes for giving me that term). It is socio-technical work in which the social and technical aspects are inseparable. It starts and ends with collaboration.
The largest single component of an engineer’s day involves social interactions with other people to enact collaboration performances. Meetings, emails, telephone conversations, text messages, document exchanges, and so on. A good proportion of the time is spent finding information.
When you ask engineers to assess how much time they spend on solitary technical work like design, calculations, problem solving and so on, the kinds of stuff we teach in universities, most disciplines report 2 to 4%. Some disciplines that involve extensive analysis, simulation or design report up to 20%.
I will try and explain why collaboration is such a large part of engineering, and why social interactions are so dominant, even at the start of an engineer’s career in jobs with predominantly technical focus.

Slide 24
I use this diagram to illustrate the components of engineering practice that came out of the studies that I performed with my students.
Engineering practice rests on a platform of technical and financial foresight: being able to make predictions and plans to enable engineering activities. It involves extensive technical collaboration performances on which results depend.
The three main results I symbolise by 3 cups of tea: my wife is addicted to tea.
1) Client satisfaction triggers payment for the work done: that’s an essential economic outcome for engineers.
2) Intrinsic achievement satisfaction, when you tell your grandchildren “See that funny little yellow thing sticking out of the side? I designed that!”
3) Success builds your reputation as an engineer which in turn leads to further work.
The collaboration performances rely on technical knowledge and understanding permeating everything. I symbolise this idea with these blue traces emanating from the right-hand corner: the technical core of engineering which stems from the education that we all strive to provide.
Yet there are two other critical supports for all engineering performances. On the left-hand side we see perception performances. And in the background what I call tacit ingenuity: knowhow that we are normally unaware of, and which enables us to create solutions.
Colouring at the base of the cones denotes the relative attention that we give these aspects in undergraduate and school education. There is considerable learning asymmetry: there is very little focus on perception skills and tacit ingenuity.

Slide 25
The slab of technical collaboration performances expands into several different sociotechnical performances which I described in my book a few years back:

Discovery learning

Informal teaching

Technical coordination

Uncertainty management

Multi-party negotiations

Slide 26
Here is my representation of the technical knowledge that graduates emerge with from engineering schools. I represent knowledge with clouds for several reasons. It’s hard to grab one: it’s hard to see how deep you are going into one, how much further you need to go, and once immersed in the cloud, it is very hard to see the world outside. I hope this analogy is meaningful for you.

Slide 27
Yet, this knowledge is only a tiny part of the knowledge that engineers rely on.

Slide 28
Here are around 35 types of technical knowledge that emerged from our research and the links between them.

Slide 29
There are also many types of organisational and business knowledge that are equally critical for engineers.

Slide 30
How can an engineering graduate learn all this when almost none was even mentioned in school?

Slide 31
Here we see her: she can cover the knowledge she learned at school.

Slide 32
Others people have specialised and cover some of the other types of knowledge.

Slide 33
Yet more people cover the other areas of knowledge she will need.

Slide 34
She collaborates with all these people to access all the other knowledge she needs. She can’t learn it all from them, not even in 10 lifetimes. Instead, she persuades them to contribute skilled performances using the knowledge they carry in their heads: a process that I have refer to as technical coordination.
This informal collaboration (a kind of informal leadership) depends on relationships: some are strong, illustrated with thick lines. Others are relatively tenuous, denoted with thin lines.

Slide 35
One of my students suggested this analogy: all these people have relationships represented by pipes between them with valves.

Slide 36
(Enlargement)
Each person can open or close the valve governing the strength of the relationship between them.
Engineering knowledge, therefore, is a multi-layered social network, a different layer for every engineering product, process or system. Traversing it takes time and social skills.
Now you can begin to appreciate the complexity of the technical collaboration on which engineering performances depend.

Slide 37
Remember the limited knowledge subset at graduation.

Slide 38
It’s such a small part…
… the narrow blue band you can see here

Slide 39
Now let’s think about the potential of education to influence workplace performances.
Many studies have shown that university performances do not seem to explain adult achievement very well.
If we focus on improving learning in the traditional technical aspects of engineering, I see very limited potential for improved engineering performances.
We could do far more by focusing on collaboration.

Slide 40
Technical engineering learning is such a small part.
Engineering performances are overwhelmingly influenced by collaboration performances. There’s far more potential to influence outcomes by focusing here. But there are many difficulties. Not the least is a curriculum which is already overcrowded.

Slide 41
Another difficulty:
So far, to my knowledge, no one has come up with evidence that contradicts this assertion I offered a couple of years ago…
Exams and written assessment methods cause students to privilege and value independent work and written communication.
To focus on collaboration, we need to balance the powerful hidden messages coming from assessment methods that contradict attempts to get students to embrace collaboration and oral communication skills.

Slide 42
Therefore, engineering education research needs to move into new spaces, particularly the early career stage for young engineers.
My books provide some starting points, but I have to admit that they leave so many questions unanswered.

Slide 43
Culture, systems and structures complicate this issue. There are many influences on engineering performances: general intelligence and learning ability, learned abilities such as emotional intelligence and languages, capabilities and expectations shaped by higher education; the culture of the host society; the culture, systems and structure of the organisation; and the interplay between structures and agency. We are only just beginning to even consider these issues.

Slide 44
To illustrate the concept of structure and agency, think about this street market. Markets like this have existed for millenia. But it takes a lot of experience, social skills, knowledge and determination to build up a loyal customer base enabling someone to earn a reliable income in markets like this.

Slide 45
In contrast, here’s a modern supermarket. The arrangement of shelves, price tags, bar codes, logistics systems, cash registers, product packaging (all incorporating heaps of embodied knowledge)… All this enables young people with little or no experience to make a decent living in an industry with extremely tight margins.
The structure enables people to be productive without much skill development. At the same time it limits their autonomy.

Slide 46
Perception skills and tacit ingenuity, two of the three pillars for engineering practice, are hardly developed at all relative to engineering science. So how is this apparent disaster averted?
Just like the supermarket, it is the structures in established engineering enterprises: formal procedures and ways of working, standards, codes, regulatory frameworks. Engineers with patchy skills (even patchy science and maths) can fit into these structures and work productively. They necessarily limit an individual engineer’s autonomy (or agency).
The lesson here is that individual performance depends as much or more on workplace culture, systems and structures as individual capabilities.
Understanding education influences will not be easy.

Slide 47
We are left with so many unanswered questions.
As Celia has pointed out in her talk, we’re not even sure how to measure competencies.
All the papers I have seen derive competencies from engineers’ or employers’ opinions. However, in our research, we found that engineers were unaware of many aspects of the work they perform. I am still to be convinced that engineers can provide reliable competency descriptions.
How well do competencies predict workplace performances? A question waiting for answers. I have not seen any studies on this.
Now,
How can we assess education interventions without understanding their effects on engineering workplace performance? I’ve seen only a few studies and the results were not encouraging.
Finally, I ask you, how can we argue for curriculum changes without answers to these questions?
My book, “Learning Engineering Practice” comes with an online appendix, a professional engineering capability framework that I developed with Engineers Australia. It sub-divides engineering performance into a couple of hundred components, each of which might be reasonably easy to teach and perhaps measure. It’s just a suggestion: you’re welcome to use it or adapt it.

Slide 48
All this is going to take a long time to sort out. So here are some short-term possibilities that you might like to try.

Slide 49
Why is global productivity not growing fast enough? In my analysis, a big part of the answer is that we stopped teaching engineers that it is their job around 50 years ago. Today we teach them that their job is to solve technical problems.

Sure, engineers solve technical problems from time to time. But so do cooks, housekeepers, business executives, lawyers, accountants. People solve problems all the time, like how to find a convenient parking spot every day they drive to work.

In fact, in our research, we found that good engineers are people who know how to avoid problems.

I argue that we a definition of engineering that shows how engineers improve productivity: one that explains the social purpose of engineering. If we fail to teach engineers that it is their job to improve human productivity, we should not be surprised if productivity does not improve.

<–>
When I started teaching listening and notetaking skills in a capstone design project course, the students asked me why they did not get help to learn that in first year. They found it really helpful and research has demonstrated the learning benefits.

<–>
Many years ago I learned that we could build students visual perception skills by getting them to learn freehand sketching. Sheryl Sorby and others have developed these ideas extensively.
<–>

From limited evidence, it is apparent that engineers are weak in critical thinking skills. Recently I had to accredit an engineering course with around 25% of the curriculum devoted to humanities and social sciences. I expected the graduates to have an excellent appreciation of critical thinking and outstanding communication skills.
<–>

It might seem surprising, that both alumni and employers still complained about communication skills. I believe they conflated communication with collaboration skills. The students and alumni that we met on the accreditation visit were extraordinarily articulate. But not necessarily good collaborators. And, furthermore, they could not distinguish critical thinking from design thinking: they were unable to explain what critical thinking really is.
<–>

Having seen it used at this university for many years, I think that the routine recording of lectures is one of the most destructive education innovations introduced in my entire career. It promotes disengagement, absence from the campus, and weakens the incentive for students to learn social skills which largely determine career success.
The design capstone course that I taught for the last few years of my career relied on classroom attendance and participation. Andrew Guzzomi will tell you how he managed to do that through the pandemic.
<–>

We need to reward students for interdependent performances. Role-play simulations are an excellent way to do that. I’m happy to elaborate more.
<–>

I argued many years ago that engineering students should learn to teach. Students learn much more from each other than they do from us. Why not build on that and, in doing so, reward them for interdependent performances? Grade them on the results achieved by the students they teach rather than their ability to recite pedagogy theories.

Slide 50
That’s it.
There’s some recent literature for your reading list. And some of my photos of Perth and other places in Western Australia to watch while you ask me some questions.