http://13.50.150.85/api.php?action=feedcontributions&feedformat=atom&user=S173876 2026-07-15T22:55:06Z User contributions MediaWiki 1.43.3 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=145315 2023-05-09T14:27:04Z

<p>S173876: /* Levels of Uncertainty – From Determinism to Indeterminacy */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Introduction: Uncertainties and Risks ==<br /> Levels of uncertainty is one of the facets of uncertainty. In the following section the basic framework of uncertainty is introduced. <br /> <br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Aleatory uncertainty is mainly about natural randomness, where epistemic is uncertainty in parameters. We will mainly be dealing with epistemic uncertainty in this article.<br /> # '''Level of uncertainty'''*. The level defines how certain we are on a spectrum between determinacy/risk to indeterminacy/ignorance. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty.<br /> <br /> == Levels of Uncertainty – From Determinism to Indeterminacy ==<br /> In the following section, a framework for the levels of uncertainty will be introduced, along with a guide on how to define the level of uncertainty a project, program or portfolio may face. This can be applied to activities, to decisions and to managerial practices, etc. <br /> <br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable). <br /> <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure 1. Derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and knowledge of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010) as seen in figure 1.<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Selecting an appropriate analysis in uncertain situations is difficult, and often times a wrong approach can lead to faulty decisions. Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyze purely risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods. By applying uncertainty levels to likelihood of events along with possibilities, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010) (See figure 2).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Figure 2. Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The matrix can be used in the following way:<br /> # Map uncertainties about possibilities and probabilities onto matrix<br /> # Decide on quadrant<br /> # Choose at minimum 2 methods to make the uncertainty more transparent<br /> <br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> <br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> <br /> '''Risk Approach'''<br /> <br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> <br /> '''Uncertainty approach''' <br /> <br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> <br /> '''Ignorance approach''' <br /> <br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> The level of uncertainty in reporting of project, program or portfolio is often not described thoroughly, or are not understood and the uncertainty level of a project, program or portfolio is interchanged between levels. This can lead to wrong interpretations of findings and analysis, leading further to poor decisions. When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013).<br /> <br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Almost all projects will use future studies to a degree, be it simulation, modelling, scenario-planning or similar. They are however often wrongly interpreted which can lead to poor management choices. Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=145301 2023-05-09T14:22:30Z

<p>S173876: /* Levels of Uncertainty – From Determinism to Indeterminacy */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Introduction: Uncertainties and Risks ==<br /> Levels of uncertainty is one of the facets of uncertainty. In the following section the basic framework of uncertainty is introduced. <br /> <br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Aleatory uncertainty is mainly about natural randomness, where epistemic is uncertainty in parameters. We will mainly be dealing with epistemic uncertainty in this article.<br /> # '''Level of uncertainty'''*. The level defines how certain we are on a spectrum between determinacy/risk to indeterminacy/ignorance. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty.<br /> <br /> == Levels of Uncertainty – From Determinism to Indeterminacy ==<br /> In the following section, a framework the levels of uncertainty will be introduced, along with a guide on how to define the level of uncertainty a project, program or portfolio may face. This can be applied to activities, to decisions and to managerial practices, etc. <br /> <br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable). <br /> <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure 1. Derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and knowledge of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010) as seen in figure 1.<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Selecting an appropriate analysis in uncertain situations is difficult, and often times a wrong approach can lead to faulty decisions. Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyze purely risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods. By applying uncertainty levels to likelihood of events along with possibilities, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010) (See figure 2).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Figure 2. Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The matrix can be used in the following way:<br /> # Map uncertainties about possibilities and probabilities onto matrix<br /> # Decide on quadrant<br /> # Choose at minimum 2 methods to make the uncertainty more transparent<br /> <br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> <br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> <br /> '''Risk Approach'''<br /> <br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> <br /> '''Uncertainty approach''' <br /> <br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> <br /> '''Ignorance approach''' <br /> <br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> The level of uncertainty in reporting of project, program or portfolio is often not described thoroughly, or are not understood and the uncertainty level of a project, program or portfolio is interchanged between levels. This can lead to wrong interpretations of findings and analysis, leading further to poor decisions. When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013).<br /> <br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Almost all projects will use future studies to a degree, be it simulation, modelling, scenario-planning or similar. They are however often wrongly interpreted which can lead to poor management choices. Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=145213 2023-05-09T14:02:02Z

<p>S173876: /* Introduction: Uncertainties and Risks */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Introduction: Uncertainties and Risks ==<br /> Levels of uncertainty is one of the facets of uncertainty. In the following section the basic framework of uncertainty is introduced. <br /> <br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Aleatory uncertainty is mainly about natural randomness, where epistemic is uncertainty in parameters. We will mainly be dealing with epistemic uncertainty in this article.<br /> # '''Level of uncertainty'''*. The level defines how certain we are on a spectrum between determinacy/risk to indeterminacy/ignorance. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty.<br /> <br /> == Levels of Uncertainty – From Determinism to Indeterminacy ==<br /> In the following section, a framework for defining levels will be introduced, along with a guide on how to define the level of uncertainty a project, program or portfolio may face. This can be applied to activities, to decisions and to managerial practices. <br /> <br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable). <br /> <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure 1. Derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and knowledge of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010) as seen in figure 1.<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Selecting an appropriate analysis in uncertain situations is difficult, and often times a wrong approach can lead to faulty decisions. Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyze purely risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods. By applying uncertainty levels to likelihood of events along with possibilities, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010) (See figure 2).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Figure 2. Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The matrix can be used in the following way:<br /> # Map uncertainties about possibilities and probabilities onto matrix<br /> # Decide on quadrant<br /> # Choose at minimum 2 methods to make the uncertainty more transparent<br /> <br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> <br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> <br /> '''Risk Approach'''<br /> <br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> <br /> '''Uncertainty approach''' <br /> <br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> <br /> '''Ignorance approach''' <br /> <br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> The level of uncertainty in reporting of project, program or portfolio is often not described thoroughly, or are not understood and the uncertainty level of a project, program or portfolio is interchanged between levels. This can lead to wrong interpretations of findings and analysis, leading further to poor decisions. When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013).<br /> <br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Almost all projects will use future studies to a degree, be it simulation, modelling, scenario-planning or similar. They are however often wrongly interpreted which can lead to poor management choices. Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=145206 2023-05-09T13:59:14Z

<p>S173876: /* Introduction: Uncertainties and Risks */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Introduction: Uncertainties and Risks ==<br /> Levels of uncertainty is one of the facets of uncertainty. In the following section the basic framework of uncertainty is introduced. <br /> <br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. The level defines how certain we are on a spectrum between determinacy/risk to indeterminacy/ignorance. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty.<br /> <br /> == Levels of Uncertainty – From Determinism to Indeterminacy ==<br /> In the following section, a framework for defining levels will be introduced, along with a guide on how to define the level of uncertainty a project, program or portfolio may face. This can be applied to activities, to decisions and to managerial practices. <br /> <br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable). <br /> <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure 1. Derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and knowledge of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010) as seen in figure 1.<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Selecting an appropriate analysis in uncertain situations is difficult, and often times a wrong approach can lead to faulty decisions. Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyze purely risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods. By applying uncertainty levels to likelihood of events along with possibilities, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010) (See figure 2).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Figure 2. Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The matrix can be used in the following way:<br /> # Map uncertainties about possibilities and probabilities onto matrix<br /> # Decide on quadrant<br /> # Choose at minimum 2 methods to make the uncertainty more transparent<br /> <br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> <br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> <br /> '''Risk Approach'''<br /> <br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> <br /> '''Uncertainty approach''' <br /> <br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> <br /> '''Ignorance approach''' <br /> <br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> The level of uncertainty in reporting of project, program or portfolio is often not described thoroughly, or are not understood and the uncertainty level of a project, program or portfolio is interchanged between levels. This can lead to wrong interpretations of findings and analysis, leading further to poor decisions. When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013).<br /> <br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Almost all projects will use future studies to a degree, be it simulation, modelling, scenario-planning or similar. They are however often wrongly interpreted which can lead to poor management choices. Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=145201 2023-05-09T13:58:08Z

<p>S173876: /* Introduction: Uncertainties and Risks */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Introduction: Uncertainties and Risks ==<br /> Levels of uncertainty is one of the facets of uncertainty. In the following section the basic framework of uncertainty is introduced. <br /> <br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty.<br /> <br /> == Levels of Uncertainty – From Determinism to Indeterminacy ==<br /> In the following section, a framework for defining levels will be introduced, along with a guide on how to define the level of uncertainty a project, program or portfolio may face. This can be applied to activities, to decisions and to managerial practices. <br /> <br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable). <br /> <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure 1. Derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and knowledge of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010) as seen in figure 1.<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Selecting an appropriate analysis in uncertain situations is difficult, and often times a wrong approach can lead to faulty decisions. Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyze purely risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods. By applying uncertainty levels to likelihood of events along with possibilities, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010) (See figure 2).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Figure 2. Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The matrix can be used in the following way:<br /> # Map uncertainties about possibilities and probabilities onto matrix<br /> # Decide on quadrant<br /> # Choose at minimum 2 methods to make the uncertainty more transparent<br /> <br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> <br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> <br /> '''Risk Approach'''<br /> <br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> <br /> '''Uncertainty approach''' <br /> <br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> <br /> '''Ignorance approach''' <br /> <br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> The level of uncertainty in reporting of project, program or portfolio is often not described thoroughly, or are not understood and the uncertainty level of a project, program or portfolio is interchanged between levels. This can lead to wrong interpretations of findings and analysis, leading further to poor decisions. When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013).<br /> <br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Almost all projects will use future studies to a degree, be it simulation, modelling, scenario-planning or similar. They are however often wrongly interpreted which can lead to poor management choices. Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=145179 2023-05-09T13:53:14Z

<p>S173876: /* Classification of expressed uncertainty-descriptions */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Introduction: Uncertainties and Risks ==<br /> Levels of Uncertainty is one of the facets of uncertainty. In the following section the basic framework of uncertainty is introduced. <br /> <br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> == Levels of Uncertainty – From Determinism to Indeterminacy ==<br /> In the following section, a framework for defining levels will be introduced, along with a guide on how to define the level of uncertainty a project, program or portfolio may face. This can be applied to activities, to decisions and to managerial practices. <br /> <br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable). <br /> <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure 1. Derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and knowledge of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010) as seen in figure 1.<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Selecting an appropriate analysis in uncertain situations is difficult, and often times a wrong approach can lead to faulty decisions. Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyze purely risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods. By applying uncertainty levels to likelihood of events along with possibilities, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010) (See figure 2).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Figure 2. Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The matrix can be used in the following way:<br /> # Map uncertainties about possibilities and probabilities onto matrix<br /> # Decide on quadrant<br /> # Choose at minimum 2 methods to make the uncertainty more transparent<br /> <br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> <br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> <br /> '''Risk Approach'''<br /> <br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> <br /> '''Uncertainty approach''' <br /> <br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> <br /> '''Ignorance approach''' <br /> <br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> The level of uncertainty in reporting of project, program or portfolio is often not described thoroughly, or are not understood and the uncertainty level of a project, program or portfolio is interchanged between levels. This can lead to wrong interpretations of findings and analysis, leading further to poor decisions. When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013).<br /> <br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Almost all projects will use future studies to a degree, be it simulation, modelling, scenario-planning or similar. They are however often wrongly interpreted which can lead to poor management choices. Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=145170 2023-05-09T13:50:41Z

<p>S173876: /* Levels of Uncertainty – From Determinism to Indeterminacy */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Introduction: Uncertainties and Risks ==<br /> Levels of Uncertainty is one of the facets of uncertainty. In the following section the basic framework of uncertainty is introduced. <br /> <br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> == Levels of Uncertainty – From Determinism to Indeterminacy ==<br /> In the following section, a framework for defining levels will be introduced, along with a guide on how to define the level of uncertainty a project, program or portfolio may face. This can be applied to activities, to decisions and to managerial practices. <br /> <br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable). <br /> <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure 1. Derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and knowledge of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010) as seen in figure 1.<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Selecting an appropriate analysis in uncertain situations is difficult, and often times a wrong approach can lead to faulty decisions. Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyze purely risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods. By applying uncertainty levels to likelihood of events along with possibilities, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010) (See figure 2).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Figure 2. Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The matrix can be used in the following way:<br /> # Map uncertainties about possibilities and probabilities onto matrix<br /> # Decide on quadrant<br /> # Choose at minimum 2 methods to make the uncertainty more transparent<br /> <br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> <br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> <br /> '''Risk Approach'''<br /> <br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> <br /> '''Uncertainty approach''' <br /> <br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> <br /> '''Ignorance approach''' <br /> <br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> The level of uncertainty in reporting of project, program or portfolio is often not described thoroughly. This can lead to wrong interpretations of findings, leading further to poor decisions. When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Almost all projects will use future studies to a degree, be it simulation, modelling, scenario-planning or similar. They are however often wrongly interpreted which can lead to poor management choices. Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=145165 2023-05-09T13:49:39Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Introduction: Uncertainties and Risks ==<br /> Levels of Uncertainty is one of the facets of uncertainty. In the following section the basic framework of uncertainty is introduced. <br /> <br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> == Levels of Uncertainty – From Determinism to Indeterminacy ==<br /> In the following section, a framework for defining levels will be introduced, along with a guide on how to define the level of uncertainty a project, program or portfolio may face. This can be applied to activities, to decisions and to managerial practices. <br /> <br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure 1. Derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and knowledge of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010) as seen in figure 1.<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Selecting an appropriate analysis in uncertain situations is difficult, and often times a wrong approach can lead to faulty decisions. Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyze purely risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods. By applying uncertainty levels to likelihood of events along with possibilities, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010) (See figure 2).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Figure 2. Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The matrix can be used in the following way:<br /> # Map uncertainties about possibilities and probabilities onto matrix<br /> # Decide on quadrant<br /> # Choose at minimum 2 methods to make the uncertainty more transparent<br /> <br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> <br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> <br /> '''Risk Approach'''<br /> <br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> <br /> '''Uncertainty approach''' <br /> <br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> <br /> '''Ignorance approach''' <br /> <br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> The level of uncertainty in reporting of project, program or portfolio is often not described thoroughly. This can lead to wrong interpretations of findings, leading further to poor decisions. When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Almost all projects will use future studies to a degree, be it simulation, modelling, scenario-planning or similar. They are however often wrongly interpreted which can lead to poor management choices. Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144623 2023-05-09T12:09:23Z

<p>S173876: /* Selection of appropriate analytical technique */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure 1. Derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and knowledge of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010) as seen in figure 1.<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Selecting an appropriate analysis in uncertain situations is difficult, and often times a wrong approach can lead to faulty decisions. Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyze purely risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods. By applying uncertainty levels to likelihood of events along with possibilities, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010) (See figure 2).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Figure 2. Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The matrix can be used in the following way:<br /> # Map uncertainties about possibilities and probabilities onto matrix<br /> # Decide on quadrant<br /> # Choose at minimum 2 methods to make the uncertainty more transparent<br /> <br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> <br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> <br /> '''Risk Approach'''<br /> <br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> <br /> '''Uncertainty approach''' <br /> <br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> <br /> '''Ignorance approach''' <br /> <br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> The level of uncertainty in reporting of project, program or portfolio is often not described thoroughly. This can lead to wrong interpretations of findings, leading further to poor decisions. When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Almost all projects will use future studies to a degree, be it simulation, modelling, scenario-planning or similar. They are however often wrongly interpreted which can lead to poor management choices. Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144617 2023-05-09T12:08:00Z

<p>S173876: /* Selection of appropriate analytical technique */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure 1. Derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and knowledge of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010) as seen in figure 1.<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Selecting an appropriate analysis in uncertain situations is difficult, and often times a wrong approach can lead to faulty decisions. Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyze purely risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods. By applying uncertainty levels to likelihood of events along with possibilities, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010) (See figure 2).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Figure 2. Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The matrix can be used in the following way:<br /> # Map uncertainties about possibilities and probabilities onto matrix<br /> # Decide on quadrant<br /> # Choose at minimum 2 methods to make the uncertainty more transparent<br /> <br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> The level of uncertainty in reporting of project, program or portfolio is often not described thoroughly. This can lead to wrong interpretations of findings, leading further to poor decisions. When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Almost all projects will use future studies to a degree, be it simulation, modelling, scenario-planning or similar. They are however often wrongly interpreted which can lead to poor management choices. Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144594 2023-05-09T12:03:11Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure 1. Derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and knowledge of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010) as seen in figure 1.<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Selecting an appropriate analysis in uncertain situations is difficult, and often times a wrong approach can lead to faulty decisions. Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010) (See figure 2).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Figure 2. Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches (See figure 2). This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> <br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> The level of uncertainty in reporting of project, program or portfolio is often not described thoroughly. This can lead to wrong interpretations of findings, leading further to poor decisions. When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Almost all projects will use future studies to a degree, be it simulation, modelling, scenario-planning or similar. They are however often wrongly interpreted which can lead to poor management choices. Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144534 2023-05-09T11:46:21Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure 1. Derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and knowledge of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010) as seen in figure 1.<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010) (See figure 2).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Figure 2. Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches (See figure 2). This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144509 2023-05-09T11:42:44Z

<p>S173876: /* The levels spectrum – from determinism to indeterminacy */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and knowledge of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144504 2023-05-09T11:41:59Z

<p>S173876: /* The levels spectrum – from determinism to indeterminacy */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> <br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144502 2023-05-09T11:41:40Z

<p>S173876: /* The levels spectrum – from determinism to indeterminacy */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: <br /> ''“Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”.'' (Enserink et. al. 2013)<br /> By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144497 2023-05-09T11:40:30Z

<p>S173876: /* The levels spectrum – from determinism to indeterminacy */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144494 2023-05-09T11:38:47Z

<p>S173876: /* The levels spectrum – from determinism to indeterminacy */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Figure derived from Warmink (2010)]]<br /> <br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144492 2023-05-09T11:37:57Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Source: Warmink (2010)]]<br /> (Figure derived from Warmink et. al. 2010)<br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Knight, F. H. (1922). Knight's Risk, Uncertainty and Profit, Risk, Uncerainty and Profit. Quarterly Journal of Economics, 36(4), 682. https://doi.org/10.2307/1884757<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144409 2023-05-09T11:24:11Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Source: Warmink (2010)]]<br /> (Figure derived from Warmink et. al. 2010)<br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144406 2023-05-09T11:23:43Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Source: Warmink (2010)]]<br /> (Figure derived from Warmink et. al. 2010)<br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> <br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> <br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2<br /> <br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> <br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> <br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> <br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> <br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> <br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144384 2023-05-09T11:19:43Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # '''Nature''' (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # '''Level of uncertainty'''*. Indeterminacy/ignorance to determinacy/risk. <br /> # '''Division of uncertainty by cognizance''' (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # '''The location/source'''*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # '''Difficulties in communication/forming consensus'''. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # '''Incorporating uncertainties with human values/subjectivity, normativity'''. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Source: Warmink (2010)]]<br /> (Figure derived from Warmink et. al. 2010)<br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # '''Risk related public engagement''' – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # '''Risk communication''' – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # '''Social movement theory''' – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # '''Exploratory scenario thinking''' – explore multiple scenarios and their consequences to the system<br /> # '''Adaptive planning''' – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # '''Decision aiding''' – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # '''Socio-organizational resilience''' – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # '''Technical/Engineering resilience''' – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144363 2023-05-09T11:17:09Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Source: Warmink (2010)]]<br /> (Figure derived from Warmink et. al. 2010)<br /> # '''Statistical uncertainty'''. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # '''Qualitative uncertainty''' – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # '''Scenario uncertainty'''. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # '''Recognized ignorance'''. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # '''Total ignorance'''. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! '''Category''' !! '''Description''' <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering future studies:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * '''What is the uncertainty about?''' (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * '''Why is there uncertainty?''' (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). '''These should be communicated separately.''' <br /> * '''Choose an expression that fits the level of uncertainty you have identified'''. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * '''Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects'''. <br /> * '''Test the effect of your communication with your audience before presenting'''. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144342 2023-05-09T11:13:28Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (Bevan, 2022), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Source: Warmink (2010)]]<br /> (Figure derived from Warmink et. al. 2010)<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # Qualitative uncertainty – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # Total ignorance. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering these practices:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> #Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> #Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> #The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> #The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> #The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> #Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> #Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> #Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> #Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> #Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> #Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> #Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144334 2023-05-09T11:11:25Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (&lt;ref&gt;Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919&lt;ref&gt;), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Source: Warmink (2010)]]<br /> (Figure derived from Warmink et. al. 2010)<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # Qualitative uncertainty – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # Total ignorance. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering these practices:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> {{reflist}}<br /> Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144305 2023-05-09T11:06:14Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (See Bevan, 2022), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Source: Warmink (2010)]]<br /> (Figure derived from Warmink et. al. 2010)<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # Qualitative uncertainty – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # Total ignorance. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering these practices:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> {{reflist}}<br /> Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144291 2023-05-09T11:03:25Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (See Bevan, 2022), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle|thumb|Source: Warmink (2010)]]<br /> (Figure derived from Warmink et. al. 2010)<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # Qualitative uncertainty – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # Total ignorance. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering these practices:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144286 2023-05-09T11:02:46Z

<p>S173876: /* Selection of appropriate analytical technique */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (See Bevan, 2022), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle]]<br /> (Figure derived from Warmink et. al. 2010)<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # Qualitative uncertainty – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # Total ignorance. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering these practices:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144284 2023-05-09T11:02:18Z

<p>S173876: /* Application */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (See Bevan, 2022), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle]]<br /> (Figure derived from Warmink et. al. 2010)<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # Qualitative uncertainty – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # Total ignorance. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle|thumb|&quot;Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature&quot;]]<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering these practices:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144275 2023-05-09T11:00:29Z

<p>S173876: /* Application */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (See Bevan, 2022), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle]]<br /> (Figure derived from Warmink et. al. 2010)<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # Qualitative uncertainty – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # Total ignorance. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|400px|right|middle]]]<br /> (Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature)<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering these practices:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations.<br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144271 2023-05-09T10:59:50Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (See Bevan, 2022), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> [[File:Chooosing Level of Uncertainty.png|400px|right|middle]]<br /> (Figure derived from Warmink et. al. 2010)<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # Qualitative uncertainty – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # Total ignorance. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|100px|right]]]<br /> (Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature)<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering these practices:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations. <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144265 2023-05-09T10:58:45Z

<p>S173876: /* The levels spectrum – from determinism to indeterminacy */</p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (See Bevan, 2022), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # Qualitative uncertainty – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # Total ignorance. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> [[File:Chooosing Level of Uncertainty.png|500px|right|middle]]<br /> (Figure derived from Warmink et. al. 2010)<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|100px|right]]]<br /> (Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature)<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering these practices:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations. <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144256 2023-05-09T10:57:00Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (See Bevan, 2022), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # Qualitative uncertainty – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # Total ignorance. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> [[File:Chooosing Level of Uncertainty.png|100px|right]]<br /> (Figure derived from Warmink et. al. 2010)<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png|100px|right]]]<br /> (Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature)<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering these practices:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations. <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144245 2023-05-09T10:55:02Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (See Bevan, 2022), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # Qualitative uncertainty – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # Total ignorance. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> [[File:Chooosing Level of Uncertainty.png]]<br /> (Figure derived from Warmink et. al. 2010)<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png]]<br /> (Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature)<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering these practices:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations. <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144243 2023-05-09T10:54:33Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (See Bevan, 2022), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # Qualitative uncertainty – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # Total ignorance. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> [[File:Chooosing Level of Uncertainty.png]]<br /> (Figure derived from Warmink et. al. 2010)<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> [[File:Uncertainty_matrix.png]]<br /> (Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature)<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering these practices:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations. <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=File:Chooosing_Level_of_Uncertainty.png&diff=144240 2023-05-09T10:53:58Z

<p>S173876: Adapted from Warmink (2010)</p> <hr /> <div>Adapted from Warmink (2010)</div>

S173876 http://13.50.150.85/index.php?title=File:Uncertainty_matrix.png&diff=144235 2023-05-09T10:52:37Z

<p>S173876: Uncertainty matrix, adapted from Stirling 2010</p> <hr /> <div>Uncertainty matrix, adapted from Stirling 2010</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=144226 2023-05-09T10:49:37Z

<p>S173876: </p> <hr /> <div><br /> WIKI ARTICLE<br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> == Uncertainties and Risks ==<br /> Uncertainty is a multifaceted creature and has been generally defined as “Any departure from the unachievable ideal of complete determinism”. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker et. al. (2003) (original 3 parts demarcated with *), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time (See Bevan, 2022), and as such consist of:<br /> # Nature (epistemic/aleatory)*. Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility. We will mainly be dealing with epistemic uncertainty in this article. <br /> # Level of uncertainty*. Indeterminacy/ignorance to determinacy/risk. <br /> # Division of uncertainty by cognizance (or lack thereof). Different types of ignorance/surprises. (See Bevan (2022) for more information). <br /> # The location/source*. Here, we refer to Oehmen &amp; Kwakkel (2023) where the location/source can be seen in management as organizational, technical or as requirement driven. <br /> # Difficulties in communication/forming consensus. This is seen as incommensurate values (i.e. not agreed upon values between stakeholders), and we will assume that projects, programs and portfolios will contain differences in these values that drive reasoning for projects. We will mainly deal with uncertainty under incommensurate values in this article. (Oehmen &amp; Kwakkel 2023)<br /> # Incorporating uncertainties with human values/subjectivity, normativity. Incommensurate values, similar to point 5.<br /> In this article, we will put our focus towards the levels of uncertainty. <br /> === The levels spectrum – from determinism to indeterminacy ===<br /> The levels of uncertainty is a way of describing the probability of uncertainties’ states within a given system (Bevan 2022). Usually, the levels are between determinacy and indeterminacy with intermediate states between the two outer cases (Walker et. al., 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements, qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink et. al., 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk (measurable and quantifiable) and uncertainty (unmeasurable), but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker et. al. (2003) developed one of the more used distinctions of the different levels of uncertainties (Bevan 2022), which gained the addition of qualitative uncertainty as proposed by Warmink et. al. (2010). The framework of level of uncertainty is however not completely agreed upon, as other distinctions of the levels are also prevalent (See Kwakkel et. al. (2010) and Oehmen &amp; Kwakkel (2023)). The list below shows the different levels described by Walker et. al. (2003) and Warmink et. al. (2010) and moves from determinacy to total ignorance (the two ends of the spectrum):<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Statistical uncertainty is as close to determinism as uncertainty can get, and mainly affecting risk. <br /> # Qualitative uncertainty – Qualitative uncertainty is defined as any uncertainty that cannot be expressed in terms of nominally measurable values. This level of uncertainty comprises opinions of experts, linguistic probabilities, and ambiguities between people.<br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). However, as Enserink et. al. (2013) writes: “… almost paradoxically, uncertainty is not merely a lack of knowledge, actually new knowledge can increase the uncertainty.”<br /> # Total ignorance. We do not know what we do not know, and surprises will arise. <br /> Understanding the level of uncertainty within projects, programs or portfolios is important as it describes how well we understand the system that we are part of. Uncertainties are however often reduced, which can have negative consequences on the decisions and management processes. As Enserink et. al. (2013) writes: “Uncertainty is something that makes decision-making difficult, and therefore it is in their [decision makers’] interest to see it reduced to probabilities and cost benefit ratios”. By reducing uncertainties like scenario uncertainty or qualitative uncertainty to statistical uncertainty, the predictions and understanding of the behaviour in the project/program/portfolio system goes from being uncertain to perceived factual. Decisions made on changed levels of uncertainty are often poor. (Enserink et. al., 2013). <br /> Choosing the level of uncertainty can be done following the questions proposed by Warmink et. al. (2010):<br /> <br /> <br /> (Figure derived from Warmink et. al. 2010)<br /> <br /> == Uncertainties in projects, program, and portfolio management ==<br /> The body of work by the project management institute (PMI) offers several techniques, approaches and guidance for dealing with risk (The Standard for Project Management, 2021, The Standard for Program Management, 2017, The Standard for Portfolio Management, 2017). Uncertainty is however less specified. In the following we will quickly sum up the general approach to risk and uncertainty management covered in these standards along with a view from literature on uncertainty practices.<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to activities/decision highly influenced by uncertainties can aid to focus further exploration to make the uncertainty more transparent and make more guided decisions. Along with this, it is advised to implement adaptability and resiliency in projects, as uncertainties will have less of an impact this way. (The Standard for Project Management, 2021)<br /> When considering if an uncertainty should be pursued to be reduced/made transparent to make more guided decisions, it is very important to consider resource management aspects such as time, staff and money (Warmink et. al. 2010). <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds” that can be used for monitoring if program activities or projects are e.g., over schedule. Following this, appropriate measures can be taken (The Standard for Program Management, 2017). However, the interdependence between projects and the longer timeframe makes deeper uncertainties have a much bigger impact than in projects. Making programs resilient to uncertainties and changes (see section “Selection of appropriate analytical technique”) will increase the likelihood of realizing benefits from the program. As programs are, to a degree, projecting what will be deemed good for the organization, society or from an engineering standpoint, these are subject of uncertainty due to the time frame (Enserink et. al. 2013). Making plural and conditional guided choices based on the level of uncertainty faced in the program can be a determining factor of success (Stirling, 2010). For guidance on how to do this, see section “Application”. <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon. It is therefore important to create an internal knowledge of uncertainty and the degree of risk that the organization is willing to embrace. (The Standard for Portfolio Management, 2017). Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio and make better decisions by knowing where the uncertainty arises from. Prediction is a key asset in portfolio management, and making transparent the level of uncertainty surrounding the portfolio will make decisions more robust. Further, the standards call for defining a risk threshold internally, but it is also adviced to embrace uncertainty, as the risk-thresholds are less likely to perceive reality under conditions of deep uncertainty (Enserink et. al., 2013, Stirling, 2010).<br /> == Application ==<br /> The application of levels of uncertainty are within several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique ===<br /> Based on the level of uncertainty project dimensions may be exposed to, the project manager can make better analysis of the parts of the system based on the level of uncertainty attained to that specific part. By applying uncertainty levels to likelihood of events along with outcomes, it is possible to decide on an appropriate means of uncertainty analysis (Stirling, 2010).<br /> <br /> <br /> (Adapted from Stirling 2010, “Keep it Complex”, originally published in Nature)<br /> The degree of unproblematic to problematic knowledge can be translated into determinism to indeterminism – i.e. the level of uncertainty. By mapping the uncertainty level onto the uncertainty matrix, it is possible to translate the level of uncertainty a project, program or portfolio may face into actionable approaches. This will make decisions and management processes much more robust against epistemological uncertainties through highlighting and making transparent what exactly we are uncertain about (Stirling, 2010). Using the categories described earlier (Statistical, scenario, and qualitative uncertainty, along with partial ignorance and recognized ignorance) may prove difficult, as the rank-order is only a helpful metaphor (Bevan, 2022). The decision of what category a project may fall into is up to the individual manager. That said, if above qualitative uncertainty, managers could generally consider knowledge as problematic. <br /> Stirling (2010) advices decision makers to move away from a pure risk driven decision making, as this will not allow for surprises to factor in. He also highlights that quantitative and qualitative methods when used to analyse only risk are too ambiguous and will not present a whole picture of the consequences of actions. Here, the advice is to move beyond risk to uncertainty, ambiguity and ignorance, and use both quantitative and qualitative methods.<br /> Managers can equally apply the uncertainty, ambiguity and ignorance methods in projects, programs and portfolio management when there is not consensus internally over the framing of possible options, contexts, outcomes, benefits or harms. When this occurs, review the degree of uncertainty on the knowledge of outcomes and possibilities, and choose both quantitative and qualitative methods to review possible actions/decisions. If the degree of uncertainty is complex due to the system the project, program or portfolio is part of, consider using several methods from both uncertainty, ambiguity and ignorance. Managers should move away from singular guidance on decisions to plural and conditional guidance, which should follow the amount of uncertainty there is faced in the project. (Stirling, 2010)<br /> Oehmen &amp; Kwakkel (2023) goes into further detail about processes to perform under uncertainties are under incommensurate values, meaning that there is not a consensus on a definitive objective to improve upon. <br /> ==== Risk Approach ====<br /> When uncertainties are at a risk level, it is especially important to consider the following three approaches:<br /> # Risk related public engagement – Public engagement can leverage collective knowledge, create buy-in and ownership, but can create anger and mistrust if not meeting quality standards and expectations. <br /> # Risk communication – Responsible risk communication must both address the subject at hand and improve discourse by using appropriate communication methods. For factors influencing this, see Oehmen &amp; Kwakkel (2023) and guidance see section “Communicating Uncertainties”.<br /> # Social movement theory – A theory to explain when and why people move from being complacent to taking action. <br /> ==== Uncertainty approach ====<br /> Approaches to robust uncertainty management practices generally follow 3 steps according to Oehmen &amp; Kwakkel (2023):<br /> # Exploratory scenario thinking – explore multiple scenarios and their consequences to the system<br /> # Adaptive planning – plans are adaptable from the outset of the project, program or portfolio in response to future changes.<br /> # Decision aiding – Having multiple stakeholders agree requires facilitation and a learning process. <br /> ==== Ignorance approach ====<br /> Addressing ignorance is especially important at the program and portfolio level, as future operating scenarios, technical limitations or similar may not be predictable. A main strategy for addressing ignorance is creating resilience (a systems capability of managing unforeseen changes) within the projects, programs and portfolios. There are two parts to resilience (as described by Oehmen &amp; Kwakkel (2023)):<br /> # Socio-organizational resilience – This can be further deconstructed as; (1) Individual and team resilience; (2) Project and organisational resilience; and (3) Supply chain and industry resilience. <br /> # Technical/Engineering resilience – [The focus is on maintaining defined functions, avoiding discontinuities, and rapidly recovering functionality to a pre-disruption state.]<br /> The Standard for Project Management (2021) guides managers on ways of increasing resilience. Examples of ways of this are (Taken from The Standard for Project Management (2021)):<br /> *Short feedback loops<br /> *Continuous learning and improvement<br /> *Ability and willingness to anticipate multiple scenarios and prepare for multiple eventualities<br /> *And more (See The Standard for Project Management (2021) for more examples)<br /> Often in managing ignorance, the “precautionary principle” will also be used, which is similar to resilience but demands action. For more information on the risk, uncertainty and ignorance approaches, see Oehmen &amp; Kwakkel (2023).<br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project workers, managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Failure to do so can have negative consequences on the project, program or portfolio management (Enserink et. al. 2013). Tenney et. al. (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, improbable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware, and it is possible for a manager to set a baseline of uncertainty reflection (Tenney et. al. 2006). (Managers can also require multiple perspectives to be highlighted (Enserink et. al., 2013)<br /> === Levels of uncertainty in future studies: What can be stated in practice ===<br /> Future studies is often used as a way of analyzing uncertainty in order to better understand the behaviour of a system as well as inform decisions (See Uncertainty Matrix). Enserink (2013) summarizes good practice when considering these practices:<br /> * Do not mix levels of uncertainty or reduce them inexplicably. This will lead to misinterpretations of the output and reduce uncertainties to risks. This may e.g., lead to scenarios to be interpreted as facts. <br /> * Include users (managers or directors) in the production of the future studies in a co-design process. This way, miscommunication of uncertainty is reduced as they are part of defining it. <br /> * In the creation of the future studies, involve non-experts.<br /> * When managing uncertainty, managers should move away from a reduction mindset (i.e. ignoring or downgrading uncertainty levels) and towards embracing uncertainty. This could be done by making decisions transparent to the uncertainty prevailing them. <br /> * The different perceptions on uncertainty should be outlined and acknowledged in models, predictions and projections. See section “Classification of expressed uncertainty-descriptions” for potential reporting requirements. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney et. al. 2006). <br /> When communicating uncertainties and their levels, use the following questions and prompts to guide your strategy (Adapted from van der Bles et. al. (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations. <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between indeterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are somewhat unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. (Bevan, 2022)<br /> * While a transition towards plural and conditional guidance when making decisions as a manager is an improvement of singular advice, the deep complexity of uncertainty and group dynamics will still be factors that cannot be escaped. The transition will however make decisions and approaches to projects, programs and portfolios more accountable and explicit towards these factors of surprise. (Stirling, 2010)<br /> * Considering levels of uncertainty in projects, programs and portfolios is not standard practice. Extending ones management practices to include this will however increase the transparency of activities and increase the understanding of the system that is managed within. It can also aid in realizing what “standard of proof” is necessary to make decisions and embracing uncertainty to a much larger extent. This is a key step in creating resilient projects, programs and portfolios. (Enserink et. al., 2013, The Standard for Project Management, 2021, Stirling, 2010). <br /> == References ==<br /> Bevan, L. D. (2022). The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change. Futures, 137, 102919. https://doi.org/10.1016/j.futures.2022.102919<br /> Walker, W. E., Harremoës, P., Romans, J., van der Sluus, J. P., van Asselt, M. B. A., Janssen, P., &amp; Krauss, M. K. V. (2003). Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466 <br /> The Standard for Portfolio Management — Fourth Edition, Standard for Portfolio Management. (2017). Standard for Portfolio Management — Fourth Edition. Project Management Institute.<br /> The Standard for Program Management — Fourth Edition, Standard for Program Management. (2017). Standard for Program Management — Fourth Edition. Project Management Institute. <br /> The standard for project management. (2021). A Guide To the Project Management Body of Knowledge (pmbok® Guide) – Seventh Edition and the Standard for Project Management (english) (pp. xxvi, 67, 274 Seiten (unknown). Project Management Institute, Inc.<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870 <br /> Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> Stirling, A. (2010). Keep it complex. Nature, 468(7327), 1029–1031. https://doi.org/10.1038/4681029a <br /> Kwakkel, J. H., Walker, W. E., &amp; Marchau, V. A. W. J. (2010). Classifying and communicating uncertainties in model-based policy analysis. International Journal of Technology, Policy and Management, 10(4), 299–315. https://doi.org/10.1504/IJTPM.2010.036918 <br /> Oehmen, J., &amp; Kwakkel, J. (2023). Risk, Uncertainty, and Ignorance in Engineering Systems Design. Handbook of Engineering Systems Design, 1–32. https://doi.org/10.1007/978-3-030-46054-9_10-2</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=134107 2023-04-09T20:36:54Z

<p>S173876: /* Uncertainties and Risks */</p> <hr /> <div><br /> <br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article also discusses expressing uncertainty levels as a state-space, which can be viewed as a proactive way of describing the level of uncertainty. The transition between levels is not clearly defined, and the project manager should be careful about moving between state spaces. The article describes five different ways to express state spaces and probability statements in terms of levels of uncertainty, ranging from full probability density function to qualitative description of the likelihood of those states. The application of levels of uncertainties is essential in several domains, such as project management, decision making, and risk analysis. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> BIG IDEA:<br /> <br /> == Uncertainties and Risks ==<br /> <br /> Uncertainty is a multifaceted creature, and there are many definitions (none which there is a broad consensus of). In this article, we choose the definition of &quot;Any departure from the unachievable ideal of complete determinism&quot;. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker (2003), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time, and as such consist of (Bevan, 2022): <br /> # Nature (epistemic/aleatory). Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility.<br /> # Levels/type. Indeterminacy/ignorance to determinacy. <br /> # Division of one’s cognizance. <br /> # The location/source<br /> # Difficulties in communication/forming consensus <br /> # Human values/subjectivity, normativity<br /> === The levels spectrum. From determinism to ignorance ===<br /> The levels of uncertainty, or scales of uncertainty, is a way of probability the probability of uncertainties states within a given system (Bevan 2022). Usually the levels are between indeterminacy and determinacy with intermediate states between the two outer cases (Walker 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements (Bradley and Dreschel (2014), qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink, 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk and uncertainty, but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker (2003) developed one of the more used distinctions of the different levels of uncertainties. This distinction in most used (Bevan 2022), and has gained the addition of qualitative uncertainty as proposed by Warmink (2010). The list below goes from determinacy to total ignorance (the two ends of the spectrum):<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Stastical uncertainty is as close to determinism as uncertainty can get. <br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Qualitative uncertainty – missing/on its way<br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). <br /> # Total ignorance. We do not know what we do not know.<br /> <br /> == Expressing uncertainty levels as a state-spaces ==<br /> Describing the level of uncertainty can be viewed in a more proactive matter as a state-space, from which the uncertainties may move (however, be aware of the limitations, see section “Limitations”). The levels which a manager may add to project information encode information about possibilities within state-spaces and their probability. Within these state-spaces, we may draw some likelihood description of found uncertainties within projects, and these may guide the project manager to evaluate the predictive capabilities of the information given to them or set about gathering more information. (Bevan, 2022)<br /> The transition between levels is not clearly defined, and thus the project manager should be careful about moving between state spaces. That said, depending on the magnitude of the uncertainty, there are some things the manager could attempt to mitigate some of the uncertainty. See section “Application”. <br /> Bevan (2022) summarizes 5 different ways to express these state-spaces and probability statements in terms of levels of uncertainties. It is here adapted to fit with the chosen description of levels of uncertainty:<br /> {| class=&quot;wikitable&quot;<br /> |+ State space Descriptions<br /> |-<br /> ! Uncertainty level !! State/possibility space description !! Likelihood estimations of states<br /> |-<br /> | Statistical uncertainty || Exhaustive yet limited set of possible states described || Full probability density function <br /> |-<br /> | Scenario uncertainty || A range of states given || Ordinal ranking of states <br /> |-<br /> | Qualitative uncertainty || Some (but not all) possible states given || Interval given<br /> |-<br /> | Recognized ignorance || Order of magnitude estimated || Fuzzy categories described (likely, unlikely, possible, etc.)<br /> |-<br /> | Partial ignorance || Direction of a trend given || Qualitative description of the likelihood of those states <br /> |}<br /> Total ignorance is not part of this state/space description, as there would be no way of describing that state space with total ignorance of said state-space.<br /> When managing uncertainties, developing appropriate reactions is key. Hence, getting an understanding of the level of uncertainty is a start point, and while project workers may not have the vocabulary between levels, using the distinction as proposed by Bevan (2022) to describe state-spaces, it should become possible to classify uncertainties in projects, programs, and portfolios, which can be further used as described in the following sections. <br /> <br /> APPLICATION<br /> == Application of Levels of uncertainties ==<br /> The application of levels of uncertainties is on several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique (Stirling 1998) ===<br /> Missing/on its way. <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Tenney, Kværner &amp; Gjerstad (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, in-probable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware. <br /> === What can be/is stated in practice (Enserink et al. 2013) ===<br /> Missing/on its way. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney, Kværner &amp; Gjerstad 2006). <br /> When communicating uncertainties and their levels, use the following questions to guide your strategy (Adapted from van der Bles (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations. <br /> === Embracing uncertainties when the arise ===<br /> Missing/on its way<br /> <br /> == Uncertainties in projects, program, and portfolio management (Following the standards provided in the course material) ==<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to risks highly influenced by uncertainties can aid to focus further exploration to minimize the uncertainty. <br /> Knowing where uncertainties arises from early in programs/projects can aid the manager to e.g. change models (if the model implies uncertainty in the given context), bla bla bla.<br /> When considering if an uncertainty should be pursued to be reduced, it is very important to consider resource management aspects such as time, staff and money (Warmink 2010)<br /> * Degree of information in projects that could be useful <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds”. However, the interdependence between projects. <br /> * Degree of information in projects that could be useful <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon [PMI Portfolio]. Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio, and make better decisions by knowing where the uncertainty arises from.<br /> * Degree of information in portfolios that could be useful <br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between inderterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are very unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. <br /> <br /> == References ==<br /> *The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change (Bevan, 2022)<br /> *Walker, W. E., Harremoës, P., Rotmans, J., Van der Sluijs, J. P., Van Asselt, M. B. A., Janssen, P. &amp; Krayer von Krauss, M. P. 2003 Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support.<br /> *PMI Portfolio standard<br /> *PMI Program standard<br /> *PMI Project standard<br /> *Van der Bles AM, van der Linden S, Freeman ALJ, Mitchell J, Galvao AB, Zaval L, Spiegelhalter DJ. 2019 Communicating uncertainty about facts, numbers and science.<br /> *Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> *Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> *Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. *Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> *Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). *Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> *Stirling, A. (1998). Risk at a turning point? Journal of Risk Research, 1(2), 97–109. https://doi.org/10.1080/136698798377204<br /> <br /> <br /> Articles that are currently also considered:<br /> *“This Is What We Don't Know”: Treating Epistemic Uncertainty in Bayesian Networks for Risk Assessment<br /> *Risk and Uncertainty Communication<br /> *van Asselt, M. &amp; Rotmans, J. 2002 Uncertainty in integrated assessment modelling. Clim. Change 54, 75–105. (doi:10.1023/A:1015783803445)<br /> *Wynne, B. 1992 Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environ. Change 2, 111–127. (doi:10.1016/0959-3780(92)90017-2)<br /> *Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk<br /> *Don’t know, can’t know: embracing deeper uncertainties when analysing risks (Spiegelhalter, 2011)<br /> *Uncertainty concepts for integrated modeling - Review and application for identifying uncertainties and uncertainty propagation pathways (Kirschner, 2021)</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=134106 2023-04-09T20:35:56Z

<p>S173876: /* Uncertainties and Risks */</p> <hr /> <div><br /> <br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article also discusses expressing uncertainty levels as a state-space, which can be viewed as a proactive way of describing the level of uncertainty. The transition between levels is not clearly defined, and the project manager should be careful about moving between state spaces. The article describes five different ways to express state spaces and probability statements in terms of levels of uncertainty, ranging from full probability density function to qualitative description of the likelihood of those states. The application of levels of uncertainties is essential in several domains, such as project management, decision making, and risk analysis. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> BIG IDEA:<br /> <br /> == Uncertainties and Risks ==<br /> <br /> Uncertainty is a multifaceted creature, and there are many definitions (none which there is a broad consensus of). In this article, we choose the definition of &quot;Any departure from the unachievable ideal of complete determinism&quot;. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker (2003), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time, and as such consist of (Bevan, 2022): <br /> # Nature (epistemic/aleatory). Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility.<br /> # Levels/type. Indeterminacy/ignorance to determinacy. <br /> # Division of one’s cognizance. <br /> # The location/source<br /> # Difficulties in communication/forming consensus <br /> # Human values/subjectivity, normativity<br /> * The levels spectrum. Determinism -&gt; Ignorance<br /> The levels of uncertainty, or scales of uncertainty, is a way of probability the probability of uncertainties states within a given system (Bevan 2022). Usually the levels are between indeterminacy and determinacy with intermediate states between the two outer cases (Walker 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements (Bradley and Dreschel (2014), qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink, 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk and uncertainty, but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker (2003) developed one of the more used distinctions of the different levels of uncertainties. This distinction in most used (Bevan 2022), and has gained the addition of qualitative uncertainty as proposed by Warmink (2010). The list below goes from determinacy to total ignorance (the two ends of the spectrum):<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Stastical uncertainty is as close to determinism as uncertainty can get. <br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Qualitative uncertainty – missing/on its way<br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). <br /> # Total ignorance. We do not know what we do not know.<br /> <br /> == Expressing uncertainty levels as a state-spaces ==<br /> Describing the level of uncertainty can be viewed in a more proactive matter as a state-space, from which the uncertainties may move (however, be aware of the limitations, see section “Limitations”). The levels which a manager may add to project information encode information about possibilities within state-spaces and their probability. Within these state-spaces, we may draw some likelihood description of found uncertainties within projects, and these may guide the project manager to evaluate the predictive capabilities of the information given to them or set about gathering more information. (Bevan, 2022)<br /> The transition between levels is not clearly defined, and thus the project manager should be careful about moving between state spaces. That said, depending on the magnitude of the uncertainty, there are some things the manager could attempt to mitigate some of the uncertainty. See section “Application”. <br /> Bevan (2022) summarizes 5 different ways to express these state-spaces and probability statements in terms of levels of uncertainties. It is here adapted to fit with the chosen description of levels of uncertainty:<br /> {| class=&quot;wikitable&quot;<br /> |+ State space Descriptions<br /> |-<br /> ! Uncertainty level !! State/possibility space description !! Likelihood estimations of states<br /> |-<br /> | Statistical uncertainty || Exhaustive yet limited set of possible states described || Full probability density function <br /> |-<br /> | Scenario uncertainty || A range of states given || Ordinal ranking of states <br /> |-<br /> | Qualitative uncertainty || Some (but not all) possible states given || Interval given<br /> |-<br /> | Recognized ignorance || Order of magnitude estimated || Fuzzy categories described (likely, unlikely, possible, etc.)<br /> |-<br /> | Partial ignorance || Direction of a trend given || Qualitative description of the likelihood of those states <br /> |}<br /> Total ignorance is not part of this state/space description, as there would be no way of describing that state space with total ignorance of said state-space.<br /> When managing uncertainties, developing appropriate reactions is key. Hence, getting an understanding of the level of uncertainty is a start point, and while project workers may not have the vocabulary between levels, using the distinction as proposed by Bevan (2022) to describe state-spaces, it should become possible to classify uncertainties in projects, programs, and portfolios, which can be further used as described in the following sections. <br /> <br /> APPLICATION<br /> == Application of Levels of uncertainties ==<br /> The application of levels of uncertainties is on several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique (Stirling 1998) ===<br /> Missing/on its way. <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Tenney, Kværner &amp; Gjerstad (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, in-probable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware. <br /> === What can be/is stated in practice (Enserink et al. 2013) ===<br /> Missing/on its way. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney, Kværner &amp; Gjerstad 2006). <br /> When communicating uncertainties and their levels, use the following questions to guide your strategy (Adapted from van der Bles (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations. <br /> === Embracing uncertainties when the arise ===<br /> Missing/on its way<br /> <br /> == Uncertainties in projects, program, and portfolio management (Following the standards provided in the course material) ==<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to risks highly influenced by uncertainties can aid to focus further exploration to minimize the uncertainty. <br /> Knowing where uncertainties arises from early in programs/projects can aid the manager to e.g. change models (if the model implies uncertainty in the given context), bla bla bla.<br /> When considering if an uncertainty should be pursued to be reduced, it is very important to consider resource management aspects such as time, staff and money (Warmink 2010)<br /> * Degree of information in projects that could be useful <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds”. However, the interdependence between projects. <br /> * Degree of information in projects that could be useful <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon [PMI Portfolio]. Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio, and make better decisions by knowing where the uncertainty arises from.<br /> * Degree of information in portfolios that could be useful <br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between inderterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are very unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. <br /> <br /> == References ==<br /> *The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change (Bevan, 2022)<br /> *Walker, W. E., Harremoës, P., Rotmans, J., Van der Sluijs, J. P., Van Asselt, M. B. A., Janssen, P. &amp; Krayer von Krauss, M. P. 2003 Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support.<br /> *PMI Portfolio standard<br /> *PMI Program standard<br /> *PMI Project standard<br /> *Van der Bles AM, van der Linden S, Freeman ALJ, Mitchell J, Galvao AB, Zaval L, Spiegelhalter DJ. 2019 Communicating uncertainty about facts, numbers and science.<br /> *Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> *Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> *Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. *Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> *Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). *Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> *Stirling, A. (1998). Risk at a turning point? Journal of Risk Research, 1(2), 97–109. https://doi.org/10.1080/136698798377204<br /> <br /> <br /> Articles that are currently also considered:<br /> *“This Is What We Don't Know”: Treating Epistemic Uncertainty in Bayesian Networks for Risk Assessment<br /> *Risk and Uncertainty Communication<br /> *van Asselt, M. &amp; Rotmans, J. 2002 Uncertainty in integrated assessment modelling. Clim. Change 54, 75–105. (doi:10.1023/A:1015783803445)<br /> *Wynne, B. 1992 Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environ. Change 2, 111–127. (doi:10.1016/0959-3780(92)90017-2)<br /> *Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk<br /> *Don’t know, can’t know: embracing deeper uncertainties when analysing risks (Spiegelhalter, 2011)<br /> *Uncertainty concepts for integrated modeling - Review and application for identifying uncertainties and uncertainty propagation pathways (Kirschner, 2021)</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=134103 2023-04-09T20:33:22Z

<p>S173876: /* Embracing uncertainties when the arise */</p> <hr /> <div><br /> <br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article also discusses expressing uncertainty levels as a state-space, which can be viewed as a proactive way of describing the level of uncertainty. The transition between levels is not clearly defined, and the project manager should be careful about moving between state spaces. The article describes five different ways to express state spaces and probability statements in terms of levels of uncertainty, ranging from full probability density function to qualitative description of the likelihood of those states. The application of levels of uncertainties is essential in several domains, such as project management, decision making, and risk analysis. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> BIG IDEA:<br /> <br /> == Uncertainties and Risks ==<br /> * Uncertainties<br /> Several definitions: Any departure from the unachievable ideal of complete determinism. <br /> * Uncertainty framework (Bevan 2022)<br /> Uncertainty is a multifaceted creature. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker (2003), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time, and as such consist of (Bevan, 2022): <br /> # Nature (epistemic/aleatory). Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility.<br /> # Levels/type. Indeterminacy/ignorance to determinacy. <br /> # Division of one’s cognizance. <br /> # The location/source<br /> # Difficulties in communication/forming consensus <br /> # Human values/subjectivity, normativity<br /> * The levels spectrum. Determinism -&gt; Ignorance<br /> The levels of uncertainty, or scales of uncertainty, is a way of probability the probability of uncertainties states within a given system (Bevan 2022). Usually the levels are between indeterminacy and determinacy with intermediate states between the two outer cases (Walker 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements (Bradley and Dreschel (2014), qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink, 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk and uncertainty, but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker (2003) developed one of the more used distinctions of the different levels of uncertainties. This distinction in most used (Bevan 2022), and has gained the addition of qualitative uncertainty as proposed by Warmink (2010). The list below goes from determinacy to total ignorance (the two ends of the spectrum):<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Stastical uncertainty is as close to determinism as uncertainty can get. <br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Qualitative uncertainty – missing/on its way<br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). <br /> # Total ignorance. We do not know what we do not know. <br /> <br /> == Expressing uncertainty levels as a state-spaces ==<br /> Describing the level of uncertainty can be viewed in a more proactive matter as a state-space, from which the uncertainties may move (however, be aware of the limitations, see section “Limitations”). The levels which a manager may add to project information encode information about possibilities within state-spaces and their probability. Within these state-spaces, we may draw some likelihood description of found uncertainties within projects, and these may guide the project manager to evaluate the predictive capabilities of the information given to them or set about gathering more information. (Bevan, 2022)<br /> The transition between levels is not clearly defined, and thus the project manager should be careful about moving between state spaces. That said, depending on the magnitude of the uncertainty, there are some things the manager could attempt to mitigate some of the uncertainty. See section “Application”. <br /> Bevan (2022) summarizes 5 different ways to express these state-spaces and probability statements in terms of levels of uncertainties. It is here adapted to fit with the chosen description of levels of uncertainty:<br /> {| class=&quot;wikitable&quot;<br /> |+ State space Descriptions<br /> |-<br /> ! Uncertainty level !! State/possibility space description !! Likelihood estimations of states<br /> |-<br /> | Statistical uncertainty || Exhaustive yet limited set of possible states described || Full probability density function <br /> |-<br /> | Scenario uncertainty || A range of states given || Ordinal ranking of states <br /> |-<br /> | Qualitative uncertainty || Some (but not all) possible states given || Interval given<br /> |-<br /> | Recognized ignorance || Order of magnitude estimated || Fuzzy categories described (likely, unlikely, possible, etc.)<br /> |-<br /> | Partial ignorance || Direction of a trend given || Qualitative description of the likelihood of those states <br /> |}<br /> Total ignorance is not part of this state/space description, as there would be no way of describing that state space with total ignorance of said state-space.<br /> When managing uncertainties, developing appropriate reactions is key. Hence, getting an understanding of the level of uncertainty is a start point, and while project workers may not have the vocabulary between levels, using the distinction as proposed by Bevan (2022) to describe state-spaces, it should become possible to classify uncertainties in projects, programs, and portfolios, which can be further used as described in the following sections. <br /> <br /> APPLICATION<br /> == Application of Levels of uncertainties ==<br /> The application of levels of uncertainties is on several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique (Stirling 1998) ===<br /> Missing/on its way. <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Tenney, Kværner &amp; Gjerstad (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, in-probable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware. <br /> === What can be/is stated in practice (Enserink et al. 2013) ===<br /> Missing/on its way. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney, Kværner &amp; Gjerstad 2006). <br /> When communicating uncertainties and their levels, use the following questions to guide your strategy (Adapted from van der Bles (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations. <br /> === Embracing uncertainties when the arise ===<br /> Missing/on its way<br /> <br /> == Uncertainties in projects, program, and portfolio management (Following the standards provided in the course material) ==<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to risks highly influenced by uncertainties can aid to focus further exploration to minimize the uncertainty. <br /> Knowing where uncertainties arises from early in programs/projects can aid the manager to e.g. change models (if the model implies uncertainty in the given context), bla bla bla.<br /> When considering if an uncertainty should be pursued to be reduced, it is very important to consider resource management aspects such as time, staff and money (Warmink 2010)<br /> * Degree of information in projects that could be useful <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds”. However, the interdependence between projects. <br /> * Degree of information in projects that could be useful <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon [PMI Portfolio]. Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio, and make better decisions by knowing where the uncertainty arises from.<br /> * Degree of information in portfolios that could be useful <br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between inderterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are very unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. <br /> <br /> == References ==<br /> *The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change (Bevan, 2022)<br /> *Walker, W. E., Harremoës, P., Rotmans, J., Van der Sluijs, J. P., Van Asselt, M. B. A., Janssen, P. &amp; Krayer von Krauss, M. P. 2003 Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support.<br /> *PMI Portfolio standard<br /> *PMI Program standard<br /> *PMI Project standard<br /> *Van der Bles AM, van der Linden S, Freeman ALJ, Mitchell J, Galvao AB, Zaval L, Spiegelhalter DJ. 2019 Communicating uncertainty about facts, numbers and science.<br /> *Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> *Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> *Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. *Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> *Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). *Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> *Stirling, A. (1998). Risk at a turning point? Journal of Risk Research, 1(2), 97–109. https://doi.org/10.1080/136698798377204<br /> <br /> <br /> Articles that are currently also considered:<br /> *“This Is What We Don't Know”: Treating Epistemic Uncertainty in Bayesian Networks for Risk Assessment<br /> *Risk and Uncertainty Communication<br /> *van Asselt, M. &amp; Rotmans, J. 2002 Uncertainty in integrated assessment modelling. Clim. Change 54, 75–105. (doi:10.1023/A:1015783803445)<br /> *Wynne, B. 1992 Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environ. Change 2, 111–127. (doi:10.1016/0959-3780(92)90017-2)<br /> *Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk<br /> *Don’t know, can’t know: embracing deeper uncertainties when analysing risks (Spiegelhalter, 2011)<br /> *Uncertainty concepts for integrated modeling - Review and application for identifying uncertainties and uncertainty propagation pathways (Kirschner, 2021)</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=134100 2023-04-09T20:32:53Z

<p>S173876: /* References */</p> <hr /> <div><br /> <br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article also discusses expressing uncertainty levels as a state-space, which can be viewed as a proactive way of describing the level of uncertainty. The transition between levels is not clearly defined, and the project manager should be careful about moving between state spaces. The article describes five different ways to express state spaces and probability statements in terms of levels of uncertainty, ranging from full probability density function to qualitative description of the likelihood of those states. The application of levels of uncertainties is essential in several domains, such as project management, decision making, and risk analysis. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> BIG IDEA:<br /> <br /> == Uncertainties and Risks ==<br /> * Uncertainties<br /> Several definitions: Any departure from the unachievable ideal of complete determinism. <br /> * Uncertainty framework (Bevan 2022)<br /> Uncertainty is a multifaceted creature. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker (2003), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time, and as such consist of (Bevan, 2022): <br /> # Nature (epistemic/aleatory). Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility.<br /> # Levels/type. Indeterminacy/ignorance to determinacy. <br /> # Division of one’s cognizance. <br /> # The location/source<br /> # Difficulties in communication/forming consensus <br /> # Human values/subjectivity, normativity<br /> * The levels spectrum. Determinism -&gt; Ignorance<br /> The levels of uncertainty, or scales of uncertainty, is a way of probability the probability of uncertainties states within a given system (Bevan 2022). Usually the levels are between indeterminacy and determinacy with intermediate states between the two outer cases (Walker 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements (Bradley and Dreschel (2014), qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink, 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk and uncertainty, but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker (2003) developed one of the more used distinctions of the different levels of uncertainties. This distinction in most used (Bevan 2022), and has gained the addition of qualitative uncertainty as proposed by Warmink (2010). The list below goes from determinacy to total ignorance (the two ends of the spectrum):<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Stastical uncertainty is as close to determinism as uncertainty can get. <br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Qualitative uncertainty – missing/on its way<br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). <br /> # Total ignorance. We do not know what we do not know. <br /> <br /> == Expressing uncertainty levels as a state-spaces ==<br /> Describing the level of uncertainty can be viewed in a more proactive matter as a state-space, from which the uncertainties may move (however, be aware of the limitations, see section “Limitations”). The levels which a manager may add to project information encode information about possibilities within state-spaces and their probability. Within these state-spaces, we may draw some likelihood description of found uncertainties within projects, and these may guide the project manager to evaluate the predictive capabilities of the information given to them or set about gathering more information. (Bevan, 2022)<br /> The transition between levels is not clearly defined, and thus the project manager should be careful about moving between state spaces. That said, depending on the magnitude of the uncertainty, there are some things the manager could attempt to mitigate some of the uncertainty. See section “Application”. <br /> Bevan (2022) summarizes 5 different ways to express these state-spaces and probability statements in terms of levels of uncertainties. It is here adapted to fit with the chosen description of levels of uncertainty:<br /> {| class=&quot;wikitable&quot;<br /> |+ State space Descriptions<br /> |-<br /> ! Uncertainty level !! State/possibility space description !! Likelihood estimations of states<br /> |-<br /> | Statistical uncertainty || Exhaustive yet limited set of possible states described || Full probability density function <br /> |-<br /> | Scenario uncertainty || A range of states given || Ordinal ranking of states <br /> |-<br /> | Qualitative uncertainty || Some (but not all) possible states given || Interval given<br /> |-<br /> | Recognized ignorance || Order of magnitude estimated || Fuzzy categories described (likely, unlikely, possible, etc.)<br /> |-<br /> | Partial ignorance || Direction of a trend given || Qualitative description of the likelihood of those states <br /> |}<br /> Total ignorance is not part of this state/space description, as there would be no way of describing that state space with total ignorance of said state-space.<br /> When managing uncertainties, developing appropriate reactions is key. Hence, getting an understanding of the level of uncertainty is a start point, and while project workers may not have the vocabulary between levels, using the distinction as proposed by Bevan (2022) to describe state-spaces, it should become possible to classify uncertainties in projects, programs, and portfolios, which can be further used as described in the following sections. <br /> <br /> APPLICATION<br /> == Application of Levels of uncertainties ==<br /> The application of levels of uncertainties is on several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique (Stirling 1998) ===<br /> Missing/on its way. <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Tenney, Kværner &amp; Gjerstad (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, in-probable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware. <br /> === What can be/is stated in practice (Enserink et al. 2013) ===<br /> Missing/on its way. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney, Kværner &amp; Gjerstad 2006). <br /> When communicating uncertainties and their levels, use the following questions to guide your strategy (Adapted from van der Bles (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations. <br /> === Embracing uncertainties when the arise ===<br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management (Following the standards provided in the course material) ==<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to risks highly influenced by uncertainties can aid to focus further exploration to minimize the uncertainty. <br /> Knowing where uncertainties arises from early in programs/projects can aid the manager to e.g. change models (if the model implies uncertainty in the given context), bla bla bla.<br /> When considering if an uncertainty should be pursued to be reduced, it is very important to consider resource management aspects such as time, staff and money (Warmink 2010)<br /> * Degree of information in projects that could be useful <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds”. However, the interdependence between projects. <br /> * Degree of information in projects that could be useful <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon [PMI Portfolio]. Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio, and make better decisions by knowing where the uncertainty arises from.<br /> * Degree of information in portfolios that could be useful <br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between inderterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are very unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. <br /> <br /> == References ==<br /> *The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change (Bevan, 2022)<br /> *Walker, W. E., Harremoës, P., Rotmans, J., Van der Sluijs, J. P., Van Asselt, M. B. A., Janssen, P. &amp; Krayer von Krauss, M. P. 2003 Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support.<br /> *PMI Portfolio standard<br /> *PMI Program standard<br /> *PMI Project standard<br /> *Van der Bles AM, van der Linden S, Freeman ALJ, Mitchell J, Galvao AB, Zaval L, Spiegelhalter DJ. 2019 Communicating uncertainty about facts, numbers and science.<br /> *Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> *Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> *Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. *Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> *Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). *Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> *Stirling, A. (1998). Risk at a turning point? Journal of Risk Research, 1(2), 97–109. https://doi.org/10.1080/136698798377204<br /> <br /> <br /> Articles that are currently also considered:<br /> *“This Is What We Don't Know”: Treating Epistemic Uncertainty in Bayesian Networks for Risk Assessment<br /> *Risk and Uncertainty Communication<br /> *van Asselt, M. &amp; Rotmans, J. 2002 Uncertainty in integrated assessment modelling. Clim. Change 54, 75–105. (doi:10.1023/A:1015783803445)<br /> *Wynne, B. 1992 Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environ. Change 2, 111–127. (doi:10.1016/0959-3780(92)90017-2)<br /> *Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk<br /> *Don’t know, can’t know: embracing deeper uncertainties when analysing risks (Spiegelhalter, 2011)<br /> *Uncertainty concepts for integrated modeling - Review and application for identifying uncertainties and uncertainty propagation pathways (Kirschner, 2021)</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=134092 2023-04-09T20:31:24Z

<p>S173876: </p> <hr /> <div><br /> <br /> == Abstract ==<br /> This wiki article discusses uncertainties and risks and the different levels of uncertainty that exist. Uncertainty is defined as any departure from the ideal of complete determinism, and it has several facets, such as nature, levels, division of one's cognizance, location/source, difficulties in communication, and human values/subjectivity. The levels of uncertainty, or scales of uncertainty, have been proposed by various scholars and are a way of measuring the probability of uncertainty states within a given system. There are several levels of uncertainty, ranging from statistical uncertainty, which can be expressed in statistical terms, to total ignorance, where we do not know what we do not know. The article also discusses expressing uncertainty levels as a state-space, which can be viewed as a proactive way of describing the level of uncertainty. The transition between levels is not clearly defined, and the project manager should be careful about moving between state spaces. The article describes five different ways to express state spaces and probability statements in terms of levels of uncertainty, ranging from full probability density function to qualitative description of the likelihood of those states. The application of levels of uncertainties is essential in several domains, such as project management, decision making, and risk analysis. The article describes how managers can use the levels of uncertainties to classify uncertainties in projects, programs, and portfolios, and evaluate the predictive capabilities of the information given to them or set about gathering more information. Developing appropriate reactions is key in managing uncertainties and risks, and getting an understanding of the level of uncertainty is a starting point.<br /> <br /> BIG IDEA:<br /> <br /> == Uncertainties and Risks ==<br /> * Uncertainties<br /> Several definitions: Any departure from the unachievable ideal of complete determinism. <br /> * Uncertainty framework (Bevan 2022)<br /> Uncertainty is a multifaceted creature. Many frameworks have been developed to explain uncertainty, with different aspects. Most frameworks base themselves on the work of Walker (2003), dealing with 3 facets of uncertainty. Additions to the frameworks have however been made over time, and as such consist of (Bevan, 2022): <br /> # Nature (epistemic/aleatory). Typically distuinguished through measurability, Meta-uncertainty, Nature, reducility.<br /> # Levels/type. Indeterminacy/ignorance to determinacy. <br /> # Division of one’s cognizance. <br /> # The location/source<br /> # Difficulties in communication/forming consensus <br /> # Human values/subjectivity, normativity<br /> * The levels spectrum. Determinism -&gt; Ignorance<br /> The levels of uncertainty, or scales of uncertainty, is a way of probability the probability of uncertainties states within a given system (Bevan 2022). Usually the levels are between indeterminacy and determinacy with intermediate states between the two outer cases (Walker 2003), but there have been changes to this original description over time, such as describing the levels as a magnitude of uncertainty, e.g. one’s ability to make judgements (Bradley and Dreschel (2014), qualitative uncertainty, e.g. expert opinions and ambiguities between people (Warmink, 2010). <br /> The original distinction between levels of uncertainties was proposed by Knight (1921) between risk and uncertainty, but quickly further developed by Keynes (1921) describing the difference as probabilities (numerical, comparable, non-comparable, uncertainty). <br /> In more recent years, Walker (2003) developed one of the more used distinctions of the different levels of uncertainties. This distinction in most used (Bevan 2022), and has gained the addition of qualitative uncertainty as proposed by Warmink (2010). The list below goes from determinacy to total ignorance (the two ends of the spectrum):<br /> # Statistical uncertainty. Any uncertainty that can be expressed in statistical terms, e.g. sampling error, measurement uncertainty with data, inaccuracy, etc. Stastical uncertainty is as close to determinism as uncertainty can get. <br /> # Scenario uncertainty. A scenario is a plausible description of how a system may act in the future, and act as an indication of a possibility. With a scenario uncertainty, mechanisms leading to specific outcomes are not well understood. It can manifest itself from e.g. different underlying assumptions from the modelers, uncertainty of driving forces in a given system, and uncertainty about the levels of changes. <br /> # Qualitative uncertainty – missing/on its way<br /> # Recognized ignorance. Uncertainty about underlying mechanisms and relationships. Scenarios are difficult to create due to this weak understanding of the system. Ignorance can be either reducible or irreducible (i.e., research can either reduce recognized ignorance or it will not be able to help). <br /> # Total ignorance. We do not know what we do not know. <br /> <br /> == Expressing uncertainty levels as a state-spaces ==<br /> Describing the level of uncertainty can be viewed in a more proactive matter as a state-space, from which the uncertainties may move (however, be aware of the limitations, see section “Limitations”). The levels which a manager may add to project information encode information about possibilities within state-spaces and their probability. Within these state-spaces, we may draw some likelihood description of found uncertainties within projects, and these may guide the project manager to evaluate the predictive capabilities of the information given to them or set about gathering more information. (Bevan, 2022)<br /> The transition between levels is not clearly defined, and thus the project manager should be careful about moving between state spaces. That said, depending on the magnitude of the uncertainty, there are some things the manager could attempt to mitigate some of the uncertainty. See section “Application”. <br /> Bevan (2022) summarizes 5 different ways to express these state-spaces and probability statements in terms of levels of uncertainties. It is here adapted to fit with the chosen description of levels of uncertainty:<br /> {| class=&quot;wikitable&quot;<br /> |+ State space Descriptions<br /> |-<br /> ! Uncertainty level !! State/possibility space description !! Likelihood estimations of states<br /> |-<br /> | Statistical uncertainty || Exhaustive yet limited set of possible states described || Full probability density function <br /> |-<br /> | Scenario uncertainty || A range of states given || Ordinal ranking of states <br /> |-<br /> | Qualitative uncertainty || Some (but not all) possible states given || Interval given<br /> |-<br /> | Recognized ignorance || Order of magnitude estimated || Fuzzy categories described (likely, unlikely, possible, etc.)<br /> |-<br /> | Partial ignorance || Direction of a trend given || Qualitative description of the likelihood of those states <br /> |}<br /> Total ignorance is not part of this state/space description, as there would be no way of describing that state space with total ignorance of said state-space.<br /> When managing uncertainties, developing appropriate reactions is key. Hence, getting an understanding of the level of uncertainty is a start point, and while project workers may not have the vocabulary between levels, using the distinction as proposed by Bevan (2022) to describe state-spaces, it should become possible to classify uncertainties in projects, programs, and portfolios, which can be further used as described in the following sections. <br /> <br /> APPLICATION<br /> == Application of Levels of uncertainties ==<br /> The application of levels of uncertainties is on several domains. Below, this article will attempt to cover some of the essential aspects of the application of levels of uncertainties. <br /> === Selection of appropriate analytical technique (Stirling 1998) ===<br /> Missing/on its way. <br /> === Classification of expressed uncertainty-descriptions ===<br /> When presented with uncertainties by project managers, experts or similar, it can be very helpful to be able to evaluate if uncertainties and their levels are thoroughly discussed and considered. Tenney, Kværner &amp; Gjerstad (2006) proposes a framework for managers to do so:<br /> {| class=&quot;wikitable&quot;<br /> |+ Uncertainty Analysis Degree<br /> |-<br /> ! Category !! Description <br /> |-<br /> | 0 || Uncertainty and their levels are not mentioned<br /> |-<br /> | X || Uncertainty and their levels are suggested, but not specifically referred to as uncertainty <br /> |-<br /> | XX || Uncertainty and their levels are indicated, sometimes estimated, but not explained or discussed <br /> |-<br /> | XXX || Uncertainty and their levels are explained and/or discussed to some degree <br /> |}<br /> With this categorization it is possible for the manager to set requirements for documentation when receiving information prior to making decisions. For example, a manager may require documentation prior to decisions to express uncertainties, at least through statements (such as probable, in-probable, etc.) but not necessarily in the wording of the specific levels of uncertainty. This way, decisions are made more transparent and aware. <br /> === What can be/is stated in practice (Enserink et al. 2013) ===<br /> Missing/on its way. <br /> === Communicating uncertainties ===<br /> Managing uncertainties in projects, programs and portfolios is not just about the prediction quality, but also about the communication and presentation of uncertainties. Decision makers must be made aware of potential impacts hidden within the uncertainties before making decisions (Tenney, Kværner &amp; Gjerstad 2006). <br /> When communicating uncertainties and their levels, use the following questions to guide your strategy (Adapted from van der Bles (2019)):<br /> * What is the uncertainty about? (Underlying hypothesis, specific numbers, categorical facts, etc.). This is also the source of uncertainty. When describing the source, consider if there are multiple or if there are relations between the sources. <br /> * Why is there uncertainty? (Natural variation, measuring difficulties, limited knowledge, disagreement between experts, etc.). <br /> * Projects will have both uncertainties directly about specific aspects of the project (e.g. a probabilistic spread, a range of scenarios, etc.?) as well as the evidence of the quality of the levels of uncertainties (e.g. why have we categorized something as a qualitative uncertainty?). These should be communicated separately. <br /> * Choose an expression that fits the level of uncertainty you have identified. This could be verbal, via media, written, or a mixture of these methods. Importantly is to consider your audience, your relationship to that audience (an vice versa), and the effect you want to impose through your communication. <br /> * Keep the level of uncertainty separate from the magnitude of evidence about the specific aspects. (i.e. you can be uncertain () about the effect of processed meat <br /> * Test the effect of your communication with your audience before presenting. Make sure that the target audience will be able to understand your descriptions and visualizations. <br /> === Embracing uncertainties when the arise ===<br /> <br /> <br /> == Uncertainties in projects, program, and portfolio management (Following the standards provided in the course material) ==<br /> === Projects ===<br /> At the project level, uncertainty is expressed through the “risk threshold”. Here, uncertainties can blur probabilities and make expressions of risk near impossible to trust. Risk response strategies may be difficult to apply if uncertainties are at a high level. Applying levels of uncertainties to risks highly influenced by uncertainties can aid to focus further exploration to minimize the uncertainty. <br /> Knowing where uncertainties arises from early in programs/projects can aid the manager to e.g. change models (if the model implies uncertainty in the given context), bla bla bla.<br /> When considering if an uncertainty should be pursued to be reduced, it is very important to consider resource management aspects such as time, staff and money (Warmink 2010)<br /> * Degree of information in projects that could be useful <br /> === Programs ===<br /> At the program level, much like at the project level, uncertainty is also expressed through “risk thresholds”. However, the interdependence between projects. <br /> * Degree of information in projects that could be useful <br /> === Portfolios ===<br /> Management of risk and uncertainties has the objective of meeting the value seeked through the portfolio while maintaining an agreed-upon level of confidence/risk in the portfolio. Uncertainty is greatest at the portfolio level as the impact of uncontrolled variables can have large implications for the organization. Risk perception is an important term here, as uncertainty can have a large impact on how risk is perceived and acted upon [PMI Portfolio]. Applying levels of uncertainties at the portfolio level may allow the decision maker to create more precise perceptions of the risks in the portfolio, and make better decisions by knowing where the uncertainty arises from.<br /> * Degree of information in portfolios that could be useful <br /> <br /> == Limitations ==<br /> * Having levels as rank-orderable is only a helpful metaphor, there are several inconsistencies within the idea. An example being that the “line” between inderterminacy and full determinacy is non-linear, although it may seem that way. The transition between levels are very unclear, and as such should not be used as an indicator of knowledge within a project, program, or portfolio (Bevan, 2022). <br /> * We may only be able to partially describe state-spaces, meaning that while we can attribute certain levels to certain parts of a system, other parts may have more indeterminate/determinate qualities that can affect the first described ones. This also means that we can feel a false sense of security in having some probabilitistic model of some events, leading to unexpected events with unknown probabilities having a much larger impact. <br /> <br /> == References ==<br /> The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change (Bevan, 2022)<br /> Walker, W. E., Harremoës, P., Rotmans, J., Van der Sluijs, J. P., Van Asselt, M. B. A., Janssen, P. &amp; Krayer von Krauss, M. P. 2003 Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support.<br /> PMI Portfolio standard<br /> PMI Program standard<br /> PMI Project standard<br /> Van der Bles AM, van der Linden S, Freeman ALJ, Mitchell J, Galvao AB, Zaval L, Spiegelhalter DJ. 2019 Communicating uncertainty about facts, numbers and science.<br /> Warmink, J. J., Janssen, J. A. E. B., Booij, M. J., &amp; Krol, M. S. (2010). Identification and classification of uncertainties in the application of environmental models. Environmental Modelling and Software, 25(12), 1518–1527. https://doi.org/10.1016/j.envsoft.2010.04.011<br /> Tennøy, A., Kværner, J., &amp; Gjerstad, K. I. (2006). Uncertainty in environmental impact assessment predictions: The need for better communication and more transparency. Impact Assessment and Project Appraisal, 24(1), 45–56. https://doi.org/10.3152/147154606781765345<br /> Enserink, B., Kwakkel, J. H., &amp; Veenman, S. (2013). Coping with uncertainty in climate policy making: (Mis)understanding scenario studies. Futures, 53, 1–12. https://doi.org/10.1016/j.futures.2013.09.006<br /> Van Der Bles, A. M., Van Der Linden, S., Freeman, A. L. J., Mitchell, J., Galvao, A. B., Zaval, L., &amp; Spiegelhalter, D. J. (2019). Communicating uncertainty about facts, numbers and science. Royal Society Open Science, 6(5), 181870. https://doi.org/10.1098/rsos.181870<br /> Stirling, A. (1998). Risk at a turning point? Journal of Risk Research, 1(2), 97–109. https://doi.org/10.1080/136698798377204<br /> <br /> <br /> Articles that are currently also considered:<br /> “This Is What We Don't Know”: Treating Epistemic Uncertainty in Bayesian Networks for Risk Assessment<br /> Risk and Uncertainty Communication<br /> van Asselt, M. &amp; Rotmans, J. 2002 Uncertainty in integrated assessment modelling. Clim. Change 54, 75–105. (doi:10.1023/A:1015783803445)<br /> Wynne, B. 1992 Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environ. Change 2, 111–127. (doi:10.1016/0959-3780(92)90017-2)<br /> Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk<br /> Don’t know, can’t know: embracing deeper uncertainties when analysing risks (Spiegelhalter, 2011)<br /> <br /> Uncertainty concepts for integrated modeling - Review and application for identifying uncertainties and uncertainty propagation pathways (Kirschner, 2021)</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=128471 2023-02-19T18:33:07Z

<p>S173876: /* Methods for modelling uncertainties */</p> <hr /> <div><br /> == Abstract ==<br /> Understanding different levels of uncertainty is important for a program, portfolio or project (PPP) manager as uncertainties can cause issues, confusion or discrepancies within PPP's. Most assessments of uncertainty is based on (potentially faulty and inadequate) assumptions, which can lead to poorly designed decisions. Understanding where the source of the uncertainty arises from can therefore aid the manager in understanding how to deal with them. Uncertainties are especially important to consider as they are often not quantifiable (and attempts to do so depends on other uncertainties that may be context based). This article seeks to highlight what different levels of uncertainties have been defined, from where they may arise, and how to manage them in a systematized manner. The article will/may contain:<br /> <br /> <br /> EDIT: The wiki deleted all my content made on sunday and did not save anything. Recovering it would take too long, so doing it another day. It looks different now, but I am not redoing it at 19:27 on a sunday. Needless to say, the levels of uncertainties will follow that which is proposed by Walker (2003) (location, type, model) with Spigelhalters as further discussion/embellishment on top of the conceptual frameworks. Application <br /> <br /> BIG IDEA:<br /> <br /> == Uncertainties and Risks ==<br /> * Definition of an uncertainty<br /> * Definition of a risk<br /> * The difference between the two<br /> * 4 types of uncertainties (as proposed by Wynne, 1992)<br /> <br /> <br /> == Levels of Uncertainties ==<br /> <br /> * The 5 levels of uncertainties (as proposed by Spiegelhalter and Riesch (2011))<br /> ** event uncertainties<br /> ** parameter uncertainties<br /> ** model based uncertainties<br /> ** acknowledged model based uncertainties<br /> ** unknown inadequacies model based uncertainties<br /> <br /> * And others distinctions (Walker (2003) on risk themes to consider (location, level, nature), van Assalt (2003) on types of uncertainties in climate governance, etc.)<br /> <br /> APPLICATION<br /> <br /> == Uncertainties in projects, program, and portfolio management (Following the standards provided in the course material) ==<br /> <br /> <br /> == Methods for modelling uncertainties ==<br /> * Bayesian models<br /> ** Different types and their implications on uncertainty<br /> * Walker (2003) identification and categorization of uncertainties (Inspired by Kirschner 2021)<br /> <br /> == Managing uncertainties when they arise ==<br /> * Communicating uncertainties<br /> <br /> LIMITATIONS<br /> <br /> == Limitations ==<br /> <br /> <br /> Articles that are considered (not part of the abstract, will be considered but not for sure part yet, just here for reference):<br /> <br /> “This Is What We Don't Know”: Treating Epistemic Uncertainty in Bayesian Networks for Risk Assessment<br /> <br /> Risk and Uncertainty Communication<br /> <br /> Walker, W. E., Harremoës, P., Rotmans, J., Van der Sluijs, J. P., Van Asselt, M. B. A., Janssen, P. &amp; Krayer von Krauss, M. P. 2003 Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support.<br /> <br /> van Asselt, M. &amp; Rotmans, J. 2002 Uncertainty in integrated assessment modelling. Clim. Change 54, 75–105. (doi:10.1023/A:1015783803445)<br /> <br /> Wynne, B. 1992 Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environ. Change 2, 111–127. (doi:10.1016/0959-3780(92)90017-2)<br /> <br /> PMI standards for project, program and portfolio management<br /> <br /> Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk<br /> <br /> The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change (Bevan, 2022)<br /> <br /> Don’t know, can’t know: embracing deeper uncertainties when analysing risks (Spiegelhalter, 2011)<br /> <br /> Uncertainty concepts for integrated modeling - Review and application for identifying uncertainties and uncertainty propagation pathways (Kirschner, 2021)</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=128466 2023-02-19T18:31:57Z

<p>S173876: </p> <hr /> <div><br /> == Abstract ==<br /> Understanding different levels of uncertainty is important for a program, portfolio or project (PPP) manager as uncertainties can cause issues, confusion or discrepancies within PPP's. Most assessments of uncertainty is based on (potentially faulty and inadequate) assumptions, which can lead to poorly designed decisions. Understanding where the source of the uncertainty arises from can therefore aid the manager in understanding how to deal with them. Uncertainties are especially important to consider as they are often not quantifiable (and attempts to do so depends on other uncertainties that may be context based). This article seeks to highlight what different levels of uncertainties have been defined, from where they may arise, and how to manage them in a systematized manner. The article will/may contain:<br /> <br /> <br /> EDIT: The wiki deleted all my content made on sunday and did not save anything. Recovering it would take too long, so doing it another day. It looks different now, but I am not redoing it at 19:27 on a sunday. Needless to say, the levels of uncertainties will follow that which is proposed by Walker (2003) (location, type, model) with Spigelhalters as further discussion/embellishment on top of the conceptual frameworks. Application <br /> <br /> BIG IDEA:<br /> <br /> == Uncertainties and Risks ==<br /> * Definition of an uncertainty<br /> * Definition of a risk<br /> * The difference between the two<br /> * 4 types of uncertainties (as proposed by Wynne, 1992)<br /> <br /> <br /> == Levels of Uncertainties ==<br /> <br /> * The 5 levels of uncertainties (as proposed by Spiegelhalter and Riesch (2011))<br /> ** event uncertainties<br /> ** parameter uncertainties<br /> ** model based uncertainties<br /> ** acknowledged model based uncertainties<br /> ** unknown inadequacies model based uncertainties<br /> <br /> * And others distinctions (Walker (2003) on risk themes to consider (location, level, nature), van Assalt (2003) on types of uncertainties in climate governance, etc.)<br /> <br /> APPLICATION<br /> <br /> == Uncertainties in projects, program, and portfolio management (Following the standards provided in the course material) ==<br /> <br /> <br /> == Methods for modelling uncertainties ==<br /> <br /> == Managing uncertainties when they arise ==<br /> * Communicating uncertainties<br /> <br /> LIMITATIONS<br /> <br /> == Limitations ==<br /> <br /> <br /> Articles that are considered (not part of the abstract, will be considered but not for sure part yet, just here for reference):<br /> <br /> “This Is What We Don't Know”: Treating Epistemic Uncertainty in Bayesian Networks for Risk Assessment<br /> <br /> Risk and Uncertainty Communication<br /> <br /> Walker, W. E., Harremoës, P., Rotmans, J., Van der Sluijs, J. P., Van Asselt, M. B. A., Janssen, P. &amp; Krayer von Krauss, M. P. 2003 Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support.<br /> <br /> van Asselt, M. &amp; Rotmans, J. 2002 Uncertainty in integrated assessment modelling. Clim. Change 54, 75–105. (doi:10.1023/A:1015783803445)<br /> <br /> Wynne, B. 1992 Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environ. Change 2, 111–127. (doi:10.1016/0959-3780(92)90017-2)<br /> <br /> PMI standards for project, program and portfolio management<br /> <br /> Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk<br /> <br /> The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change (Bevan, 2022)<br /> <br /> Don’t know, can’t know: embracing deeper uncertainties when analysing risks (Spiegelhalter, 2011)<br /> <br /> Uncertainty concepts for integrated modeling - Review and application for identifying uncertainties and uncertainty propagation pathways (Kirschner, 2021)</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=126789 2023-02-18T10:43:18Z

<p>S173876: </p> <hr /> <div><br /> == Abstract ==<br /> Understanding different levels of uncertainty is important for a program, portfolio or project (PPP) manager as uncertainties can cause issues, confusion or discrepancies within PPP's. Most assessments of uncertainty is based on (potentially faulty and inadequate) assumptions, which can lead to poorly designed decisions. Understanding where the source of the uncertainty arises from can therefore aid the manager in understanding how to deal with them. Uncertainties are especially important to consider as they are often not quantifiable (and attempts to do so depends on other uncertainties that may be context based). This article seeks to highlight what different levels of uncertainties have been defined, from where they may arise, and how to manage them in a systematized manner. The article will/may contain:<br /> <br /> <br /> BIG IDEA:<br /> <br /> == Uncertainties and Risks ==<br /> * Definition of an uncertainty<br /> * Definition of a risk<br /> * The difference between the two<br /> * 4 types of uncertainties (as proposed by Wynne, 1992)<br /> <br /> <br /> == Levels of Uncertainties ==<br /> <br /> * The 5 levels of uncertainties (as proposed by Spiegelhalter and Riesch (2011))<br /> ** event uncertainties<br /> ** parameter uncertainties<br /> ** model based uncertainties<br /> ** acknowledged model based uncertainties<br /> ** unknown inadequacies model based uncertainties<br /> <br /> * And others distinctions (Walker (2003) on risk themes to consider (location, level, nature), van Assalt (2003) on types of uncertainties in climate governance, etc.)<br /> <br /> APPLICATION<br /> <br /> == Uncertainties in projects, program, and portfolio management (Following the standards provided in the course material) ==<br /> <br /> == Methods for modelling uncertainties ==<br /> <br /> == Communicating uncertainties ==<br /> <br /> == Managing uncertainties when they arise ==<br /> <br /> <br /> LIMITATIONS<br /> <br /> == Limitations ==<br /> <br /> <br /> Articles that are considered (not part of the abstract, will be considered but not for sure part yet, just here for reference):<br /> <br /> “This Is What We Don't Know”: Treating Epistemic Uncertainty in Bayesian Networks for Risk Assessment<br /> <br /> Risk and Uncertainty Communication<br /> <br /> Walker, W. E., Harremoës, P., Rotmans, J., Van der Sluijs, J. P., Van Asselt, M. B. A., Janssen, P. &amp; Krayer von Krauss, M. P. 2003 Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support.<br /> <br /> van Asselt, M. &amp; Rotmans, J. 2002 Uncertainty in integrated assessment modelling. Clim. Change 54, 75–105. (doi:10.1023/A:1015783803445)<br /> <br /> Wynne, B. 1992 Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environ. Change 2, 111–127. (doi:10.1016/0959-3780(92)90017-2)<br /> <br /> PMI standards for project, program and portfolio management<br /> <br /> Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk<br /> <br /> The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change (Bevan, 2022)<br /> <br /> Don’t know, can’t know: embracing deeper uncertainties when analysing risks (Spiegelhalter, 2011)</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=126788 2023-02-18T10:42:50Z

<p>S173876: /* Abstract */</p> <hr /> <div><br /> == Abstract ==<br /> Understanding different levels of uncertainty is important for a program, portfolio or project (PPP) manager as uncertainties can cause issues, confusion or discrepancies within PPP's. Most assessments of uncertainty is based on (potentially faulty and inadequate) assumptions, which can lead to poorly designed decisions. Understanding where the source of the uncertainty arises from can therefore aid the manager in understanding how to deal with them. Uncertainties are especially important to consider as they are often not quantifiable (and attempts to do so depends on other uncertainties that may be context based). This article seeks to highlight what different levels of uncertainties have been defined, from where they may arise, and how to manage them in a systematized manner. The article will/may contain:<br /> <br /> <br /> BIG IDEA:<br /> <br /> == Uncertainties and Risks ==<br /> * Definition of an uncertainty<br /> * Definition of a risk<br /> * The difference between the two<br /> * 4 types of uncertainties (as proposed by Wynne, 1992)<br /> <br /> <br /> == Levels of Uncertainties ==<br /> <br /> * The 5 levels of uncertainties (as proposed by Spiegelhalter and Riesch (2011))<br /> * * event uncertainties<br /> * * parameter uncertainties<br /> * * model based uncertainties<br /> * * acknowledged model based uncertainties<br /> * * unknown inadequacies model based uncertainties<br /> <br /> * And others distinctions (Walker (2003) on risk themes to consider (location, level, nature), van Assalt (2003) on types of uncertainties in climate governance, etc.)<br /> <br /> APPLICATION<br /> <br /> == Uncertainties in projects, program, and portfolio management (Following the standards provided in the course material) ==<br /> <br /> == Methods for modelling uncertainties ==<br /> <br /> == Communicating uncertainties ==<br /> <br /> == Managing uncertainties when they arise ==<br /> <br /> <br /> LIMITATIONS<br /> <br /> == Limitations ==<br /> <br /> <br /> Articles that are considered (not part of the abstract, will be considered but not for sure part yet, just here for reference):<br /> <br /> “This Is What We Don't Know”: Treating Epistemic Uncertainty in Bayesian Networks for Risk Assessment<br /> <br /> Risk and Uncertainty Communication<br /> <br /> Walker, W. E., Harremoës, P., Rotmans, J., Van der Sluijs, J. P., Van Asselt, M. B. A., Janssen, P. &amp; Krayer von Krauss, M. P. 2003 Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support.<br /> <br /> van Asselt, M. &amp; Rotmans, J. 2002 Uncertainty in integrated assessment modelling. Clim. Change 54, 75–105. (doi:10.1023/A:1015783803445)<br /> <br /> Wynne, B. 1992 Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environ. Change 2, 111–127. (doi:10.1016/0959-3780(92)90017-2)<br /> <br /> PMI standards for project, program and portfolio management<br /> <br /> Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk<br /> <br /> The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change (Bevan, 2022)<br /> <br /> Don’t know, can’t know: embracing deeper uncertainties when analysing risks (Spiegelhalter, 2011)</div>

S173876 http://13.50.150.85/index.php?title=Articles_Spring_Term_2023&diff=124427 2023-02-12T10:57:54Z

<p>S173876: /* Overview of 2023 Wiki Collections */</p> <hr /> <div>=Overview of 2023 Wiki Collections=<br /> <br /> {| class=&quot;wikitable sortable&quot;<br /> |+Spring Term 2023 Wiki Collections<br /> |Group 23<br /> |Foteini <br /> |Pange<br /> |s222872<br /> |New? / revised?<br /> |[[Stakeholder and Social Network Analysis]]<br /> |-<br /> |-<br /> |Group 14<br /> |Ali<br /> |Abdul-Ghani<br /> |s184640<br /> |New<br /> |[[name]]<br /> |-<br /> |-<br /> |Group 19<br /> |Fritzdorf<br /> |Julia<br /> |s226721<br /> |New<br /> |[[Perception filters for learning and communicating]]<br /> |-<br /> <br /> |-<br /> |Group 14<br /> |Fatima<br /> |Sheraz<br /> |s220010<br /> |New<br /> |[[Innovation Management]]<br /> |-<br /> <br /> <br /> |-<br /> |Group 23<br /> |Papadaki Fanioudaki<br /> |Maria<br /> |s230214<br /> |New? / revised?<br /> |[[Cost build up estimation in projects]]<br /> |-<br /> |-<br /> |Group 20<br /> |Manuela<br /> |Vazquez<br /> |s222648<br /> |New<br /> |[[WBS - Work Breakdown Structure]]<br /> |-<br /> |-<br /> |Group 23<br /> |Michalis<br /> |Pianos<br /> |s220253<br /> |New? / revised?<br /> |[[Change requests in Project Management]]<br /> |-<br /> |-<br /> |Group 24<br /> |Josefine <br /> |Rolver<br /> |s183652<br /> |New<br /> |[[Managing stakeholders through persona]]<br /> |-<br /> |-<br /> |Group 17<br /> |Sofie<br /> |Heide-Ottosen<br /> |s193943<br /> |New<br /> |[[Groups vs teams]]<br /> |-<br /> |-<br /> |Group 19<br /> |Nikolaos <br /> |Kavros<br /> |s213235<br /> |New? / revised?<br /> |[[Project Analytics]]<br /> |-<br /> |-<br /> |Group <br /> |Kristoffer<br /> |Skjøde Rander<br /> |s175027<br /> |New? / revised?<br /> |[[Strategic Misrepresentation]]<br /> |-<br /> |-<br /> |Group 9<br /> |Aleksander<br /> |Moczko<br /> |s214147<br /> |New<br /> |[[Satisficing]]<br /> |-<br /> |-<br /> |Group name<br /> |Anna<br /> |Diedrichsen<br /> |s193956<br /> |New? / revised?<br /> |[[Conflict resolution through MBTI]]<br /> |-<br /> |-<br /> |Group 24<br /> |Martin<br /> |Sørensen<br /> |s183676<br /> |New<br /> |[[Matrix organizations]]<br /> |-<br /> |-<br /> |Group name<br /> |Mohamed<br /> |Anshur<br /> |s185115<br /> |New? / revised?<br /> |[[Article name]]<br /> |- <br /> |- <br /> |Group name<br /> |Pablo Andres<br /> |Gomez<br /> |s223631<br /> |New? / revised?<br /> |[[Net Present Value (NPV) - Discounted cash flow]]<br /> |-<br /> |-<br /> |Group 20<br /> |Helena Søndberg<br /> |Svendsen<br /> |s223135<br /> |New<br /> |[[Concurrent Engineering]]<br /> |-<br /> |-<br /> |Group 21<br /> |Johannes Nicolás<br /> |Wildfeuer<br /> |s213244<br /> |New<br /> |[[When – the scientific secrets of perfect timing]]<br /> |-<br /> |-<br /> |Group 29<br /> |Sunneva<br /> |Gaardlykke<br /> |s163822<br /> |New<br /> |[[Eisenhower Decision Matrix]]<br /> |-<br /> |-<br /> |Group 21<br /> |Nicolas<br /> |Stephan<br /> |S226693<br /> |New<br /> |[[Diversity]]<br /> |-<br /> |-<br /> |Group 21<br /> |Ingo<br /> |Baur<br /> |S226768<br /> |New<br /> |[[Self-Awareness!]]<br /> |-<br /> |-<br /> |Group 9<br /> |Mikkel<br /> |Bjerregaard<br /> |S184965<br /> |New? / revised?<br /> |[[The HEXACO Model of Personality Structure]]<br /> |-<br /> |-<br /> |Group 11<br /> |Anton<br /> |Reiling<br /> |s222572<br /> |New<br /> |[[Data-Driven Decision-Making under Uncertainty]]<br /> |-<br /> |-<br /> |Group X<br /> |Mathias<br /> |Iversen<br /> |S174750<br /> |New? / revised?<br /> |[[EQ vs. IQ]]<br /> |-<br /> |-<br /> |Group X<br /> |Marcus<br /> |Christiansen<br /> |s194506<br /> |New? / revised?<br /> |[[Project organization]]<br /> |-<br /> |-<br /> |Group 29<br /> |Paraskevi<br /> |Keramari<br /> |s230249<br /> |New? / revised?<br /> |[[The Hawthorne studies]]<br /> |-<br /> |-<br /> |Group X<br /> |Christian<br /> |Honoré<br /> |s175077<br /> |New? / revised?<br /> |[[Circular Economy in Project Management]]<br /> |-<br /> |-<br /> |Group 12<br /> |Mansoor<br /> |Samadi<br /> |s174669<br /> |New<br /> |[[MCDM-APH method in decision making]]<br /> |-<br /> |-<br /> |Group X<br /> |Sebastian Christian Harhoff<br /> |Pieters<br /> |s184149<br /> |New? / revised?<br /> |[[FMEA]]<br /> |-<br /> |-<br /> |Group 12<br /> |Lillian<br /> |Nygaard<br /> |S174726<br /> |New? / revised?<br /> |[[The Stage Gate process: A powerful method used for efficient project management]]<br /> |-<br /> |-<br /> |Group name<br /> |Lukas<br /> |Bonitz<br /> |s230263<br /> |New? / revised?<br /> |[[(WBS) - Work Breakdown Structure]]<br /> |-<br /> |-<br /> |Group 31<br /> |Francesco<br /> |Magnani<br /> |s230227<br /> |New? / revised?<br /> |[[Boosting Team Engagement through Gamification: Mitigating the Effects of Social Loafing]]<br /> |-<br /> |-<br /> |Group name<br /> |Gaia<br /> |Sassone<br /> |s222532<br /> |New? / revised?<br /> |[[Applying the Pareto Principle in Risk Management]]<br /> |-<br /> |-<br /> |Group X<br /> |Ísabella Rós<br /> |Ingimundardóttir<br /> |s222538<br /> |New? / revised?<br /> |[[Personality Types and Leadership]]<br /> |-<br /> |-<br /> |Group TBD<br /> |Sasha<br /> |Mantel<br /> |s222742<br /> |New? / revised?<br /> |[[Project Stakeholder Management]]<br /> |-<br /> |-<br /> |Group 23<br /> |Julie<br /> |Hustoft<br /> |s194359<br /> |New<br /> |[[The Benefits of Self-Awareness]]<br /> |-<br /> |-<br /> |Group TBD<br /> |Mikkel<br /> |Anderson<br /> |s184230<br /> |New? / revised?<br /> |[[Continuous Integration/Continuous Delivery (CI/CD)]]<br /> |-<br /> |-<br /> |Group 12<br /> |Emma <br /> |Bigum<br /> |s160048<br /> |New<br /> |[[Lippitt-Knoster Complex Change Management]]<br /> |-<br /> |-<br /> |Group 22<br /> |Ingibjörg <br /> |Einarsdóttir<br /> |s223228<br /> |New? / revised?<br /> |[[Mediating conflicts and controversy]]<br /> |-<br /> |-<br /> |Group 4<br /> |Peter <br /> |Aggerholm<br /> |s170890<br /> |New? / revised?<br /> |[[When: The Scientific Secrets of Perfect Timing]]<br /> |-<br /> |-<br /> |Group 36<br /> |Hekla Mist <br /> |Valgeirsdóttir<br /> |s223229<br /> |New? / revised?<br /> |[[Management and leadership differences]]<br /> |-<br /> |-<br /> |Group 20<br /> |Theis Rosenkvist <br /> |Sørensen<br /> |s195768<br /> |New? / revised?<br /> |[[Managing threats and opportunities in risk treatment]]<br /> |-<br /> |-<br /> |Group 9<br /> |Julie Amanda<br /> |Busch<br /> |s190974<br /> |New<br /> |[[Utilizing Value Functions for Evaluating the Performance of Project Alternatives]]<br /> |-<br /> |-<br /> |Group 6<br /> |Martin<br /> |Fehst<br /> |s222575<br /> |New? / revised?<br /> |[[Leading multidisciplinary Project Teams]]<br /> |-<br /> |-<br /> |Group 11<br /> |Eric Vincent<br /> |Vong<br /> |s222594<br /> |New<br /> |[[PDCA cycle]]<br /> |-<br /> |-<br /> |Group 23<br /> |Efthymios <br /> |Samaras<br /> |s223292<br /> |New? / revised?<br /> |[[Benefit Cost Ratio (BCR)]]<br /> |-<br /> |-<br /> |Group 1<br /> |Cirkeline <br /> |Bräuner<br /> |s193974<br /> |Revised<br /> |[[Lessons learned]]<br /> |-<br /> |-<br /> |Group 2<br /> |Hrannar <br /> |Þórarinsson<br /> |s222536<br /> |New<br /> |[[Balanced scorecard: connecting the performance measures]]<br /> |-<br /> |-<br /> |Group 15<br /> |Mar <br /> |Rivera Hedo<br /> |s222520<br /> |New? / revised?<br /> |[[The Importance of Psychological Safety in Team Development]]<br /> |-<br /> |-<br /> |Group 36<br /> |Thordis <br /> |Ragnarsdottir<br /> |s222535<br /> |New? / revised?<br /> |[[Sources of conflicts]]<br /> |-<br /> |-<br /> |Group 5<br /> |Oriol<br /> |Solans Ormo<br /> |s222527<br /> |New? / revised?<br /> |[[Sustainability Issue]]<br /> |-<br /> |-<br /> |Group 15<br /> |Naïl<br /> |Mulatier<br /> |s222590<br /> |New<br /> |[[Addressing diversity and inclusion in a short-term project]]<br /> <br /> |-<br /> |-<br /> |Group 22<br /> |Valdís <br /> |Sigurdardóttir<br /> |s2223305<br /> |New<br /> |[[Challenges and opportunities regarding diversity within teams work]]<br /> |-<br /> |-<br /> |Group 12<br /> |Sophia <br /> |Fjeldsøe<br /> |s184141<br /> |New<br /> |[[Stakeholder Mapping]]<br /> <br /> <br /> |-<br /> |-<br /> |Group 29<br /> |Miriam <br /> |Khader<br /> |s194717<br /> |New<br /> |[[Leadership development through MBTI]]<br /> |-<br /> |-<br /> |Group 5<br /> |Maria<br /> |Arrillaga Tarazona<br /> |s222708<br /> |New? / revised?<br /> |[[Program Evaluation and Review Technique (PERT)]]<br /> <br /> |-<br /> |-<br /> |Group 24<br /> |Josefine <br /> |Steinfurth<br /> |s183648<br /> |New<br /> |[[Project based organisations]]<br /> <br /> |-<br /> |-<br /> |Group 5<br /> |Patricio Alejandro <br /> |Fabro<br /> |s220269<br /> |New<br /> |[[Metrics in Portfolio management]]<br /> <br /> |-<br /> |-<br /> |Group 10<br /> |Jurek Noah <br /> |von Petersdorff-Campen<br /> |s222570<br /> |New<br /> |[[Competency Mapping for Project Management]]<br /> <br /> |-<br /> |-<br /> |Group 15<br /> |Inke<br /> |Lindberg<br /> |s223191<br /> |New<br /> |[[Post-occupancy evaluation (POE)]]<br /> |-<br /> |-<br /> |Group 16<br /> |Ahmet<br /> |Ibre<br /> |s222442<br /> |New<br /> |[[Social loafing in multidisciplinary student groups]]<br /> <br /> |-<br /> |-<br /> |Group 16<br /> |Marie-Therese <br /> |Guidje<br /> |s180863<br /> |New? / revised?<br /> |[[Sustainable Project management]]<br /> |-<br /> |-<br /> |Group 12<br /> |Peter <br /> |Andresen<br /> |s190517<br /> |New<br /> |[[Project performance with Value Stream management]]<br /> |-<br /> |-<br /> |Group 3<br /> |Annas<br /> |Jadi <br /> |s194831<br /> |New/ Revised<br /> |[[Key performance indicators for portfolio management]]<br /> |-<br /> |-<br /> |Group X<br /> |Fabio<br /> |Colombo<br /> |s212492<br /> |New/ Revised<br /> |[[Robust Decision Making: better decisions under uncertainty]]<br /> |-<br /> |-<br /> |Group 20<br /> |Rune<br /> |Knudsen<br /> |s193983<br /> |New<br /> |[[Pooled, Sequential &amp; Reciprocal dependence]]<br /> |-<br /> |-<br /> |Group 24<br /> |Mia<br /> |Hemmingsen<br /> |s183639<br /> |New/ Revised<br /> |[[Seven stages of change]]<br /> |-<br /> |-<br /> |Group 1<br /> |Christina<br /> |Kjær<br /> |s183504<br /> |New/ Revised<br /> |[[Belbin's team roles, 6C model]]<br /> |-<br /> |-<br /> |Group 21<br /> |Jakob<br /> |Berling<br /> |s230256<br /> |New<br /> |[[Stress]]<br /> |-<br /> |-<br /> |Group 1<br /> |Joakim <br /> |Cayouette<br /> |s193978<br /> |Revised<br /> |[[Forecasting and estimation techniques]]<br /> |-<br /> |-<br /> |Group 30<br /> |Amalie<br /> |Pedersen<br /> |s193967<br /> |New<br /> |[[Satisficing]]<br /> -<br /> |-<br /> |Group 30<br /> |Filip Furbo<br /> |Enevoldsen<br /> |s193967<br /> |New<br /> |[[Optimism bias in teams]]<br /> |-<br /> |-<br /> |Group 23<br /> |Nicolai<br /> |Frausing<br /> |s175034<br /> |New<br /> |[[Benefit Realization Management]]<br /> |-<br /> |-<br /> |Group<br /> |Emma<br /> |Egelund<br /> |s171516<br /> |Revised<br /> |[[Network Planning in Project Management]]<br /> |-<br /> |-<br /> |Group 21<br /> |Henrik<br /> |Svensson<br /> |s230245<br /> |New<br /> |[[Participatory Design]]<br /> |-<br /> |-<br /> |Group 11<br /> |Luisa Fernanda<br /> |Salazar Rivera<br /> |s222401<br /> |New<br /> |[[Building Effective Work Breakdown Structures (WBS)]]<br /> |-<br /> |-<br /> |Group 15<br /> |Michele <br /> |Ferrante<br /> |s220282<br /> |New<br /> |[[Safety risk management in construction sites]]<br /> |-<br /> |-<br /> |Group 4<br /> |Helena <br /> |Mladenovski<br /> |s183367<br /> | New<br /> |[[Situational Leadership Theory (SLT)]]<br /> |-<br /> |-<br /> |Group 20<br /> |Rolf<br /> |Wollesen<br /> |s193972<br /> | New<br /> |[[Risk Burn Down Chart]]<br /> |-<br /> |-<br /> |Group 6<br /> |Gianluca <br /> |Santinon<br /> |s222630<br /> |New<br /> |[[The Rolling-Wave planning technique]]<br /> |-<br /> |-<br /> |Group X<br /> |Hanna <br /> |Rakel Bjarnadóttir<br /> |s223414<br /> |New<br /> |[[Strength and weakness of sustainable management]]<br /> |-<br /> |-<br /> |Group 5<br /> |Marta <br /> |Santos<br /> |s226759<br /> |New<br /> |[[Risk Response Plan]]<br /> |-<br /> |-<br /> |Group 16<br /> |Hendrik<br /> |Stegers<br /> |s230258<br /> |New<br /> |[[Communication Theories in Project Management]]<br /> |-<br /> |-<br /> |Group X<br /> |Jiafei<br /> |Xia<br /> |s213084<br /> |New<br /> |[[Employee Loyalty Management]]<br /> |-<br /> |-<br /> |Group 15<br /> |Mathéo<br /> |Torris<br /> |s223233<br /> |New<br /> |[[Risk and Mitigation Plan]]<br /> |-<br /> |-<br /> |Group 8<br /> |Frederik<br /> |Høgdal<br /> |s173876<br /> |New<br /> |[[Levels of uncertainties]]<br /> |-<br /> |-</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=124418 2023-02-12T10:52:06Z

<p>S173876: /* Abstract */</p> <hr /> <div><br /> == Abstract ==<br /> Understanding different levels of uncertainty is important for a program, portfolio or project (PPP) manager as uncertainties can cause issues, confusion or discrepancies within PPP's. Most assessments of uncertainty is based on (potentially faulty and inadequate) assumptions, which can lead to poorly designed decisions. Understanding where the source of the uncertainty arises from can therefore aid the manager in understanding how to deal with them. Uncertainties are especially important to consider as they are often not quantifiable (and attempts to do so depends on other uncertainties that may be context based). This article seeks to highlight what different levels of uncertainties have been defined, from where they may arise, and how to manage them in a systematized manner. The article will/may contain:<br /> <br /> <br /> BIG IDEA:<br /> <br /> * The difference between risks and uncertainties<br /> <br /> * Differing views on the sources/reasons for uncertainties, considereing:<br /> <br /> ** The 5 levels of uncertainties as proposed by Spiegelhalter and Riesch (2011) (event uncertainties, parameter uncertainties, model based uncertainties, acknowledged model based uncertainties, and unknown inadequacies model based uncertainties)<br /> <br /> ** 4 types of uncertainties (Wynne, 1992)<br /> <br /> ** And others (Walker (2003) on risk themes to consider (location, level, nature), van Assalt (2003) on types of uncertainties in climate governance, etc.)<br /> <br /> APPLICATION<br /> <br /> * Uncertainties in projects, program, and portfolio management (Following the standards provided in the course material)<br /> <br /> * Methods for modelling uncertainties<br /> <br /> * Communicating uncertainties<br /> <br /> * Managing uncertainties when they arise<br /> <br /> <br /> LIMITATIONS<br /> <br /> * Limitations<br /> <br /> <br /> Articles that are considered (not part of the abstract, will be considered but not for sure part yet, just here for reference):<br /> <br /> “This Is What We Don't Know”: Treating Epistemic Uncertainty in Bayesian Networks for Risk Assessment<br /> <br /> Risk and Uncertainty Communication<br /> <br /> Walker, W. E., Harremoës, P., Rotmans, J., Van der Sluijs, J. P., Van Asselt, M. B. A., Janssen, P. &amp; Krayer von Krauss, M. P. 2003 Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support.<br /> <br /> van Asselt, M. &amp; Rotmans, J. 2002 Uncertainty in integrated assessment modelling. Clim. Change 54, 75–105. (doi:10.1023/A:1015783803445)<br /> <br /> Wynne, B. 1992 Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environ. Change 2, 111–127. (doi:10.1016/0959-3780(92)90017-2)<br /> <br /> PMI standards for project, program and portfolio management<br /> <br /> Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk<br /> <br /> The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change (Bevan, 2022)<br /> <br /> Don’t know, can’t know: embracing deeper uncertainties when analysing risks (Spiegelhalter, 2011)</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=123824 2023-02-10T16:37:11Z

<p>S173876: </p> <hr /> <div><br /> == Abstract ==<br /> Levels of uncertainty is important for a Program, portfolio and project management toolbox as all assessments of uncertainty is based on (potentially faulty and inadequate) assumptions. Uncertainties are especially important to consider as they are not quantifiable (and attempts to do so depends on other uncertainties that may be context based). This article seeks to highlight what different levels of uncertainties there may exist, and from where they may arise. The article may contain:<br /> <br /> <br /> BIG IDEA:<br /> <br /> * The difference between risks and uncertainties<br /> <br /> * Differing views on the sources/reasons for uncertainties, considereing:<br /> <br /> ** The 5 levels of uncertainties as proposed by Spiegelhalter and Riesch (2011) (event uncertainties, parameter uncertainties, model based uncertainties, acknowledged model based uncertainties, and unknown inadequacies model based uncertainties)<br /> <br /> ** 4 types of uncertainties (Wynne, 1992)<br /> <br /> ** And others (Walker (2003) on risk themes to consider (location, level, nature), van Assalt (2003) on types of uncertainties in climate governance, etc.)<br /> <br /> APPLICATION<br /> <br /> * Uncertainties in projects, program, and portfolio management (Following the standards provided in the course material)<br /> <br /> * Methods for modelling uncertainties<br /> <br /> * Communicating uncertainties<br /> <br /> * Managing uncertainties when they arise<br /> <br /> <br /> LIMITATIONS<br /> <br /> * Limitations<br /> <br /> <br /> Articles that are considered (not part of the abstract, will be considered but not for sure part yet, just here for reference):<br /> <br /> “This Is What We Don't Know”: Treating Epistemic Uncertainty in Bayesian Networks for Risk Assessment<br /> <br /> Risk and Uncertainty Communication<br /> <br /> Walker, W. E., Harremoës, P., Rotmans, J., Van der Sluijs, J. P., Van Asselt, M. B. A., Janssen, P. &amp; Krayer von Krauss, M. P. 2003 Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support.<br /> <br /> van Asselt, M. &amp; Rotmans, J. 2002 Uncertainty in integrated assessment modelling. Clim. Change 54, 75–105. (doi:10.1023/A:1015783803445)<br /> <br /> Wynne, B. 1992 Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environ. Change 2, 111–127. (doi:10.1016/0959-3780(92)90017-2)<br /> <br /> PMI standards for project, program and portfolio management<br /> <br /> Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk<br /> <br /> The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change (Bevan, 2022)<br /> <br /> Don’t know, can’t know: embracing deeper uncertainties when analysing risks (Spiegelhalter, 2011)</div>

S173876 http://13.50.150.85/index.php?title=Levels_of_uncertainties&diff=123823 2023-02-10T16:33:47Z

<p>S173876: </p> <hr /> <div>Abstract: <br /> Levels of uncertainty is important for a Program, portfolio and project management toolbox as all assessments of uncertainty is based on (potentially faulty and inadequate) assumptions. Uncertainties are especially important to consider as they are not quantifiable (and attempts to do so depends on other uncertainties that may be context based). This article seeks to highlight what different levels of uncertainties there may exist, and from where they may arise. The article may contain:<br /> <br /> BIG IDEA:<br /> <br /> - The difference between risks and uncertainties<br /> <br /> - Differing views on the sources/reasons for uncertainties, considereing:<br /> <br /> o The 5 levels of uncertainties as proposed by Spiegelhalter and Riesch (2011) (event uncertainties, parameter uncertainties, model based uncertainties, acknowledged model based uncertainties, and unknown inadequacies model based uncertainties)<br /> <br /> o 4 types of uncertainties (Wynne, 1992)<br /> <br /> o And others (Walker (2003) on risk themes to consider (location, level, nature), van Assalt (2003) on types of uncertainties in climate governance, etc.)<br /> <br /> APPLICATION<br /> <br /> - Uncertainties in projects, program, and portfolio management (Following the standards provided in the course material)<br /> <br /> - Methods for modelling uncertainties<br /> <br /> - Communicating uncertainties<br /> <br /> - Managing uncertainties when they arise<br /> <br /> LIMITATIONS<br /> <br /> - Limitations<br /> <br /> Articles that are considered (not part of the abstract, will be considered but not for sure part yet, just here for reference):<br /> <br /> “This Is What We Don't Know”: Treating Epistemic Uncertainty in Bayesian Networks for Risk Assessment<br /> <br /> Risk and Uncertainty Communication<br /> <br /> Walker, W. E., Harremoës, P., Rotmans, J., Van der Sluijs, J. P., Van Asselt, M. B. A., Janssen, P. &amp; Krayer von Krauss, M. P. 2003 Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support.<br /> <br /> van Asselt, M. &amp; Rotmans, J. 2002 Uncertainty in integrated assessment modelling. Clim. Change 54, 75–105. (doi:10.1023/A:1015783803445)<br /> <br /> Wynne, B. 1992 Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environ. Change 2, 111–127. (doi:10.1016/0959-3780(92)90017-2)<br /> <br /> PMI standards for project, program and portfolio management<br /> <br /> Handbook of Risk Theory: Epistemology, Decision Theory, Ethics, and Social Implications of Risk<br /> <br /> The ambiguities of uncertainty: A review of uncertainty frameworks relevant to the assessment of environmental change (Bevan, 2022)<br /> <br /> Don’t know, can’t know: embracing deeper uncertainties when analysing risks (Spiegelhalter, 2011)</div>

S173876