Raising risk awareness in infrastructure tenders
Due to the COVID-19 crisis the PhD defence of Jeroen van der Meer will take place online (until further notice).
The PhD defence can be followed by a live stream.
Jeroen van der Meer is a PhD student in the research group Construction Management and Engineering (CME). His supervisor is prof.dr. G.P.M.R. Dewulf from the Faculty of Engineering Technology (ET).
Constructing new or renovating existing infrastructure is necessary to keep the condition of infrastructure at the appropriate level. The authorities responsible for this infrastructure, issue public tenders to select contractors for the necessary construction work. In recent years, these types of construction projects have been regularly procured through integrated contracts in which a contractor becomes responsible for completion of multiple project phases. This encourages contractors to include life-cycle-oriented design optimizations in their bid. For example, optimizations in the buildability are possible if a contractor is responsible for both the design and construction of a project. The contractor must take into account the long-term effects of the optimizations in his search for an economically viable solution. Contractors must make design decisions to find a solution that incorporates the conflicting needs and requirements. This often involves designing several alternatives, many of which have different levels of abstraction and are based on the client's preferred design or on the functional requirements set.
Both the abstraction and the level of detail of contractual requirements, the number of requirements, the required integral and multidisciplinary approach and the increased importance of stakeholder management increase the complexity of both the tender, construction and maintenance phases of infrastructure projects. Contractors are trying to deal with the increased complexity of projects through the use of systems engineering as a design methodology, to ensure conformity to the specifications and requirements of their design and decision-making processes. The purpose of systems engineering is to gain a better understanding of customer demand and use this understanding to improve decision making throughout the life cycle. Multi-criteria decision analysis (MCDA) is commonly used to support decision making throughout the entire systems engineering process. MCDA supports conflict resolution and compliance with stakeholder needs, requirements, and preconditions. At the same time, the interactions between requirements, subsystems and preconditions are full of risk and uncertainty because detailed design information, resources and time are scarce in a tender. Consciously managing risk in tenders is essential for preparing a bid since the decisions made in this phase can have significant consequences for both the schedule and the costs. The decisions can have consequences for both the result of the effort made, for example a relatively high bid in case of overestimation of the risks, and for the result after realization, for example a relatively low bid in case of underestimation of the risks. Risk is defined as the extent to which there is uncertainty about whether significant and/or disappointing outcomes of decisions can be realized. A prerequisite for assessing risk is to be aware of the risks and to perceive a decision as risky. In literature substantial effort has been made to improve the risk assessment, but there is a lack of assessment approaches that can support the impact of risks on the different project objectives. Moreover, the available assessment tools in literature suffer from low usability in practice as professionals often rely on their experience and knowledge when making decisions. Using experience and knowledge to assess the impact of risks raises the question to what extent contractors are aware of the role of risk in making design decisions and how risk awareness can be raised. The increased complexity also affects the quality of decisions when they are made based on knowledge, experience and intuition. The quality of decisions made during tenders can be assessed by their actual outcome or based on the process of making a decision. The emphasis in this study is on the latter. The design space of projects has grown in such a way that multiple disciplines are required to reach a solution. The possible alternatives within this multidisciplinary and integrated perspective have grown accordingly and do not correspond with the engineers' frame of reference. In addition to the risk awareness of contractors, it is therefore also important to take into account the risk perception of engineers.
The aim of this dissertation is to increase the understanding about the risk awareness of contractors in integrated design decisions during the tender phase of complex infrastructure projects. It also aims at suggesting possible direction for raising risk awareness of contractors when making integrated design decisions. The central research question is: To what extent are contractors aware of risks when making design decisions in infrastructure tenders and how can risk awareness be raised? This main question is divided into four sub-questions which make up the chapters of this dissertation. The first sub-question addresses the challenges in making design decisions when systems engineering is applied in infrastructure tenders. The second sub-question is about the suitability of MCDA to ensure decision quality. The third sub-question provides insight into the decision-making behaviour of engineers by altering the risk representation in an MCDA tool. The last sub-question combines these insights and alters the decision-making process to raise risk awareness.
Chapter 2 explores the challenges in making design decisions when systems engineering and multi-criteria-analysis techniques are applied in a large infrastructure tender. Based on a tender for the reconstruction of a main traffic junction, it becomes clear that contractors apply systems engineering because they believe it reduces the complexity of projects and they have structured their design processes accordingly. However, it is still a challenge for contractors to deal with uncertainties in their decision-making process due to the low level of specifications provided. This is partly caused by the limitation in time, capacity and feasibility to perform additional research during the tender phase. As a result, contractors struggle to design a solution that will not only persuade the client but will also deliver an economically viable solution. The decision-making process therefore benefits from dealing with design uncertainty more explicitly. Making design uncertainty explicit is a first step that allows for finding methods that provide early understanding of the impact of design decisions.
Based on the challenges that contractors have in identifying uncertainties and incorporating these in their decision-making, chapter 3 examines whether MCDA is suitable to ensure decision quality of the decision-making process given several alternatives. Whether MCDA is also suitable for indicating the correctness of a solution within all possible solutions is beyond the scope of this study. MCDA structures the different solutions of the decision-making problem as well as the considerations and preferences of the stakeholders. As such, the use of MCDA enhances the quality of the decision-making process. High-quality decisions are characterized by a decision-process based on an appropriate frame, creative and feasible alternatives, reliable and unbiased information, desired outcomes, the logic by which the decision was made, and commitment of all stakeholders. The suitability of MCDA to ensure the quality of design decisions has been investigated in a case study. The case study shows that an MCDA defines the “what” in terms of structuring the decision problem, but not “how” this should be done. It also shows that decision-makers rely on their experience and knowledge when making decisions. An explicit consideration of decision quality can support the “how” by defining each criterion and evaluating whether the quality of the available information is aligned with the defined problem. Chapter 3 thus contributes to the application of MCDA by showing that 1) decision-making in tenders is based on the experience and knowledge of the engineers involved and 2) inappropriately used MCDA tools and methods can give the impression of soundly underpinned evaluation of alternatives is while the uncertainties are neglected, leading to premature decisions of low quality.
The practical application of systems engineering and MCDA shows that decisions in infrastructure tenders are made by relying on the knowledge, insight, experience and intuition of engineers. Risks associated with the different alternatives are often overlooked. A prerequisite for the assessment of the risks is that engineers are aware of the risks and that, in addition, a process is in place to bring together the individual perceptions into a shared assessment of the risks. Chapter 4 examines whether adjustments to the representation of risks in a trade-off matrix (ToM) can influence the risk perception of engineers. In comparable studies this is researched based on experimental settings, in a controlled environment and with fictitious and simplified choice alternatives. In contrast to these studies, this research is conducted in an environment similar to an actual tender and with reality-based design alternatives. By combining an experiment followed by in-depth interviews, it is not only possible to test the relationship between risk presentation, risk perception and decision-making, but it is also possible to better explain the test results. The results show a limited effect of risk representation on risk perception and decision-making behaviour. Engineers' knowledge, experience, and perceptions remain dominant for decision making. These findings contradict with studies that use simple experimental manipulations in a controlled laboratory environment. The reproducibility of experimental results in practice is therefore challenged. Based on the interviews, the insignificant relationships between risk perception and the decision-making behaviour of engineers are explained by (1) the effectiveness of the applied choice architecture and (2) the real-life setting of the experiment.
Chapter 5 examines the possibility of increasing risk awareness by adjusting the decision-making process. The decision-making process has been analysed during an ongoing tender by applying a design science research approach. Three possible interventions have been identified that increase risk awareness:
- Change the format of the ToM by including a description of the criteria and using a general list of criteria to identify criteria that match the characteristics of the tender.
- Explicitly link the identified project risks to the criteria and assign a bandwidth value (the most likely, the minimum and the maximum value).
- Evaluate the quality of the decision-making process by assigning a value to the quality elements "relevant and unbiased information", "desired results" and "logic" of the decision-making process.
These interventions should increase the transparency and rationality of decisions and, by doing so, raise risk awareness. The interventions were implemented in the decision-making process in which the operation of the interventions were validated in a workshop setting. The three interventions triggered and structured discussions and helped to gain insight into the perceptions and reasoning of professionals. The resulting general design rules represent the first ingredients towards an action-oriented theory for creating risk awareness in the project context:
- Jointly defining criteria to increase common understanding of the criteria and use of relevant information.
- Highlighting or visualizing uncertainty in the scoring of criteria to trigger discussion about the risks.
- Reflecting on the decision-making process by evaluating decision quality.
These design rules play an important role in the stepwise process that defines “how” an MCDA should be used during an infrastructure tender. Applying the design rules stimulates discussion within the tender team and this discussion reveals the underlying reasoning and interpretations of criteria and increase risk awareness. This leads to a more transparent problem understanding among engineers and to more rational choices, even though these choices are based on engineers' experiences. Conducting an MCDA in a construction tender context requires knowledge of both “what” is required in terms of structuring the decision problem and “how” in terms of guiding the decision-making process.
