Robust Decision Making (RDM) - Revision history

S212830: /* Limitations */

2022-03-15T21:27:03Z

Limitations

← Older revision		Revision as of 21:27, 15 March 2022
Line 124:		Line 124:


	Another limitation of RDM is the complexity of the tools needed to create and explore the different scenarios.<ref name="research gate">Agusdinata, Datu Buyung. (2008). Exploratory modeling and analysis: a promising method to deal with deep uncertainty. </ref> . First, these tools can be difficult to use and require an expert who should be part of the ~~project~~. Second, it may seem that suitable scenario generators do not exist for very specific types of projects. In this case, they must be implemented before proceeding with RDM. And this requires significant human and financial resources.		Another limitation of RDM is the complexity of the tools needed to create and explore the different scenarios.<ref name="research gate">Agusdinata, Datu Buyung. (2008). Exploratory modeling and analysis: a promising method to deal with deep uncertainty. </ref> . First, these tools can be difficult to use and require an expert who should be part of the process. Second, it may seem that suitable scenario generators do not exist for very specific types of projects. In this case, they must be implemented before proceeding with RDM. And this requires significant human and financial resources.

	== Annotated Bibliography==		== Annotated Bibliography==

S212830: /* Analyze and characterize the database produced */

2022-03-15T21:23:05Z

Analyze and characterize the database produced

← Older revision		Revision as of 21:23, 15 March 2022
Line 104:		Line 104:
	=== Analyze and characterize the database produced ===		=== Analyze and characterize the database produced ===

	This step consists of applying the methodology defined in ~~point e~~. The expected result is an identification and visualization of the key factors in the scenarios for which your strategies do or do not work. The goal here is not to make a conclusion but to create a support that will help decision makers make the trade-off analysis described below.		This step consists of applying the methodology defined in the section "Define a strategy of exploration and analyze of the data produced by the simulation modeling". The expected result is an identification and visualization of the key factors in the scenarios for which your strategies do or do not work. The goal here is not to make a conclusion but to create a support that will help decision makers make the trade-off analysis described below.

	=== Realize a tradeoff Analysis ===		=== Realize a tradeoff Analysis ===

S212830: /* Run the simulation */

2022-03-15T21:22:22Z

Run the simulation

← Older revision		Revision as of 21:22, 15 March 2022
Line 100:		Line 100:
	=== Run the simulation ===		=== Run the simulation ===

	This step consists only of applying the strategy defined in ~~step D~~. The expected result is the database mentioned above and a way to visualize the path through the different futures.		This step consists only of applying the strategy defined in the action "Define a plan for conducting the simulation modeling". The expected result is the database mentioned above and a way to visualize the path through the different futures.

	=== Analyze and characterize the database produced ===		=== Analyze and characterize the database produced ===

S212830: /* Define a plan for conducting the simulation modeling */

2022-03-15T21:20:38Z

Define a plan for conducting the simulation modeling

← Older revision		Revision as of 21:20, 15 March 2022
Line 90:		Line 90:
	In addition, your simulations will generate a lot of data. In order to make it usable for the next steps, it is crucial to define how this data will be collected and stored. The knowledge and tools used in the field of data analysis can be applied here.		In addition, your simulations will generate a lot of data. In order to make it usable for the next steps, it is crucial to define how this data will be collected and stored. The knowledge and tools used in the field of data analysis can be applied here.

	A final point that needs to be considered during the planning phase is the computational power required by the scenario generation. As mentioned in the ~~previous article (you can find the~~ part ~~about computing power in one of the previous articles)~~, this can require a lot of computing performance. And even if computers are making more and more progress, too many scenarios can exceed the capacities you have or create a very long waiting time. It is therefore necessary to anticipate these aspects and to reduce the complexity of scenario generations if necessary.		A final point that needs to be considered during the planning phase is the computational power required by the scenario generation. As mentioned in the collaboration Human/Machine part, this can require a lot of computing performance. And even if computers are making more and more progress, too many scenarios can exceed the capacities you have or create a very long waiting time. It is therefore necessary to anticipate these aspects and to reduce the complexity of scenario generations if necessary.

	More information can be found in the references dedicated to exploratory modeling. <ref name="research gate">Agusdinata, Datu Buyung. (2008). Exploratory modeling and analysis: a promising method to deal with deep uncertainty. </ref>		More information can be found in the references dedicated to exploratory modeling. <ref name="research gate">Agusdinata, Datu Buyung. (2008). Exploratory modeling and analysis: a promising method to deal with deep uncertainty. </ref>

S212830: /* Define a plan for conducting the simulation modeling */

2022-03-15T21:17:53Z

Define a plan for conducting the simulation modeling

← Older revision		Revision as of 21:17, 15 March 2022
Line 88:		Line 88:
	The simulation model is a key factor in RDM. There are two things that must be considered during the simulation.<ref name="Shaping the Next One Hundred Years">Lempert, Robert J., Steven W. Popper, and Steven C. Bankes, Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis, Santa Monica, Calif.: RAND Corporation, MR-1626-RPC, 2003. As of March 05, 2022: https://www.rand.org/pubs/monograph_reports/MR1626.html </ref>. First, the model must be able to generate a wide range of potential futures based on the uncertainties defined above. Second, the model must provide access to the different steps it creates in all potential futures. This is essential to allow humans to understand the impact of the assumptions made at each stage on the strategies. The first point requires a scenario generator and the second requires exploratory modeling software. It is then necessary to choose a software adapted to the domain and the scope of your project.		The simulation model is a key factor in RDM. There are two things that must be considered during the simulation.<ref name="Shaping the Next One Hundred Years">Lempert, Robert J., Steven W. Popper, and Steven C. Bankes, Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis, Santa Monica, Calif.: RAND Corporation, MR-1626-RPC, 2003. As of March 05, 2022: https://www.rand.org/pubs/monograph_reports/MR1626.html </ref>. First, the model must be able to generate a wide range of potential futures based on the uncertainties defined above. Second, the model must provide access to the different steps it creates in all potential futures. This is essential to allow humans to understand the impact of the assumptions made at each stage on the strategies. The first point requires a scenario generator and the second requires exploratory modeling software. It is then necessary to choose a software adapted to the domain and the scope of your project.

	In addition, your simulations will generate a lot of data. In order to make it usable for the next steps, it is crucial to define how this data will be collected and stored.		In addition, your simulations will generate a lot of data. In order to make it usable for the next steps, it is crucial to define how this data will be collected and stored. The knowledge and tools used in the field of data analysis can be applied here.

	A final point that needs to be considered during the planning phase is the computational power required by the scenario generation. As mentioned in the previous article (you can find the part about computing power in one of the previous articles), this can require a lot of computing performance. And even if computers are making more and more progress, too many scenarios can exceed the capacities you have or create a very long waiting time. It is therefore necessary to anticipate these aspects and to reduce the complexity of scenario generations if necessary.		A final point that needs to be considered during the planning phase is the computational power required by the scenario generation. As mentioned in the previous article (you can find the part about computing power in one of the previous articles), this can require a lot of computing performance. And even if computers are making more and more progress, too many scenarios can exceed the capacities you have or create a very long waiting time. It is therefore necessary to anticipate these aspects and to reduce the complexity of scenario generations if necessary.

S212830: /* Step by Step methodology */

2022-03-15T21:11:51Z

Step by Step methodology

← Older revision		Revision as of 21:11, 15 March 2022
Line 54:		Line 54:
	== Step by Step methodology ==		== Step by Step methodology ==

	The purpose of this section is to provide a detailed guide for the reader to apply ~~MDM~~ to a new project, program or portfolio management situation. Each subsection presents a step in the process and briefly discusses the tools useful for that section. The order of presentation is chronological. The different steps are based on the methodology defined by Robert Lempert. <ref name="open knowledge"> Robert Lempert, Nidhi Kalra, Suzanne Peyraud, Zhimin Mao, Sinh Bach Tan, Dean Cira, Alexander Lotsch (2013) Ensuring Robust Flood Risk Management in Ho Chi Minh City https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref>		The purpose of this section is to provide a detailed guide for the reader to apply RDM to a new project, program or portfolio management situation. Each subsection presents a step in the process and briefly discusses the tools useful for that section. The order of presentation is chronological. The different steps are based on the methodology defined by Robert Lempert. <ref name="open knowledge"> Robert Lempert, Nidhi Kalra, Suzanne Peyraud, Zhimin Mao, Sinh Bach Tan, Dean Cira, Alexander Lotsch (2013) Ensuring Robust Flood Risk Management in Ho Chi Minh City https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref>

S212830: /* RDM fondations */

2022-03-15T21:09:37Z

RDM fondations

← Older revision		Revision as of 21:09, 15 March 2022
Line 44:		Line 44:
	When all of these characteristics are known, they can be passed on to decision makers who will perform a trade-off analysis to find a robust strategy. The strategies can then be modified or combined to find a robust one. That is to say “ones that meet multiple objectives over many scenarios “<ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. The process described above can be repeated several times to reveal new strategies and result in an even stronger decision.		When all of these characteristics are known, they can be passed on to decision makers who will perform a trade-off analysis to find a robust strategy. The strategies can then be modified or combined to find a robust one. That is to say “ones that meet multiple objectives over many scenarios “<ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. The process described above can be repeated several times to reveal new strategies and result in an even stronger decision.

	Because RDM is useful for defining strategy, it is used early in the project/program or portfolio life cycle.		Because RDM is useful for defining strategy, it should be used early in the project/program or portfolio life cycle.

	=== Collaboration Human/machine ===		=== Collaboration Human/machine ===

S212830: /* RDM fondations */

2022-03-15T21:06:33Z

RDM fondations

← Older revision		Revision as of 21:06, 15 March 2022
Line 36:		Line 36:
	The first is the set of data relevant to assessing whether a strategy is suitable for a specific future regarding the achievement of predefined goals. The data used to say whether a strategy is suitable or not for a situation really depends on that situation. There is an extensive literature that describes how to choose KPIs and gives examples of data that may be relevant. RDM can use these results and methodologies to include them in the process.		The first is the set of data relevant to assessing whether a strategy is suitable for a specific future regarding the achievement of predefined goals. The data used to say whether a strategy is suitable or not for a situation really depends on that situation. There is an extensive literature that describes how to choose KPIs and gives examples of data that may be relevant. RDM can use these results and methodologies to include them in the process.

	The second type of data is a description of the path used to generate the range of possible futures. That is, all the characteristics of one path that are relevant to differentiate it from another. This data is mainly generated by coupling algorithms capable of generating scenarios and software capable of exploring these scenarios. <ref name="Robust decision making" >https://doi.org/10.1016/j.envsoft.2016.09.017</ref> - see ~~steps-~~by~~-steps~~ section ~~for more info~~.		The second type of data is a description of the path used to generate the range of possible futures. That is, all the characteristics of one path that are relevant to differentiate it from another. This data is mainly generated by coupling algorithms capable of generating scenarios and software capable of exploring these scenarios. <ref name="Robust decision making" >https://doi.org/10.1016/j.envsoft.2016.09.017</ref> - For more information, see the Steps by Steps section.

S212830: /* RDM fondations */

2022-03-15T21:04:24Z

RDM fondations

← Older revision		Revision as of 21:04, 15 March 2022
Line 31:		Line 31:
	=== RDM fondations ===		=== RDM fondations ===

	As mentioned in the literature <ref name="open knowledge"> Robert Lempert, Nidhi Kalra, Suzanne Peyraud, Zhimin Mao, Sinh Bach Tan, Dean Cira, Alexander Lotsch (2013) Ensuring Robust Flood Risk Management in Ho Chi Minh City https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref> , RDM does not use computer models and data as a predictive tool to anticipate the best estimate of the future. Instead, ~~RDMs use~~ the principle of exploratory modeling. <ref name="research gate">Agusdinata, Datu Buyung. (2008). Exploratory modeling and analysis: a promising method to deal with deep uncertainty. </ref> . This means that a large number of possible futures with many different paths are generated to test the proposed strategies. The objective is not to rank the plausible scenarios but to explore as many as possible. Comparisons of the pre-decided strategies with a wide range of scenarios are then used to create a database of key performance indicators and outcomes. In this database, there are two main types of data.		As mentioned in the literature <ref name="open knowledge"> Robert Lempert, Nidhi Kalra, Suzanne Peyraud, Zhimin Mao, Sinh Bach Tan, Dean Cira, Alexander Lotsch (2013) Ensuring Robust Flood Risk Management in Ho Chi Minh City https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref> , RDM does not use computer models and data as a predictive tool to anticipate the best estimate of the future. Instead, RDM is based on the principle of exploratory modeling <ref name="research gate">Agusdinata, Datu Buyung. (2008). Exploratory modeling and analysis: a promising method to deal with deep uncertainty. </ref> . This means that a large number of possible futures with many different paths are generated to test the proposed strategies. The objective is not to rank the plausible scenarios but to explore as many as possible. Comparisons of the pre-decided strategies with a wide range of scenarios are then used to create a database of key performance indicators and outcomes. In this database, there are two main types of data.

S212830 at 20:45, 15 March 2022

2022-03-15T20:45:24Z

← Older revision		Revision as of 20:45, 15 March 2022
Line 27:		Line 27:


	The fields of application that we can ~~find~~ are numerous. Here is a non-exhaustive list: <ref>Robert Lempert, Steven Popper, Steve Bankes. ROBUST DECISIONMAKING (RDM) https://millennium-project.org/wp-content/uploads/2020/02/22-Robust-Decisionmaking.pdf </ref> :defense, higher education, insurance, science and technology planning, counter-terrorism... In the current state of knowledge, RDM is very often used when a project, program or portfolio needs to include climate change in the decision. <ref name="ecotoolbox">http://eprints.lse.ac.uk/68318/1/Conway_Barriers%20and%20opportunities%20.pdf </ref> .		The fields of application that can be found are numerous. Here is a non-exhaustive list: <ref>Robert Lempert, Steven Popper, Steve Bankes. ROBUST DECISIONMAKING (RDM) https://millennium-project.org/wp-content/uploads/2020/02/22-Robust-Decisionmaking.pdf </ref> :defense, higher education, insurance, science and technology planning, counter-terrorism... In the current state of knowledge, RDM is very often used when a project, program or portfolio needs to include climate change in the decision. <ref name="ecotoolbox">http://eprints.lse.ac.uk/68318/1/Conway_Barriers%20and%20opportunities%20.pdf </ref> .

	=== RDM fondations ===		=== RDM fondations ===
Line 48:		Line 48:
	=== Collaboration Human/machine ===		=== Collaboration Human/machine ===

	An important point of RDM is that it is based on human-machine collaboration. <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. The strength of this method is that it combines the power of human decision making with the computational power of computers. Computers are used to generate and explore many different scenarios which are in turn used to stress the strategies devised by humans. When it comes to deep uncertainties with a large number of hypotheses to explore, computing power is absolutely necessary to cope with the mass of data to be processed. On the other hand, the decision-making part and the creation of potential trade-offs then rely solely on the decision-makers who can use the results of the previous computational steps as support. This mix between computer science and human intelligence implies the use of a different set of tools. As ~~we can see from~~ the ~~different~~ applications of the RDM method, it only defines the general type of tool you need for each step. The exact tools to use depend on the context and must be defined for each new project. A very quick overview and advice on the type of tools to use are detailed in the "step by step" section.		An important point of RDM is that it is based on human-machine collaboration. <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. The strength of this method is that it combines the power of human decision making with the computational power of computers. Computers are used to generate and explore many different scenarios which are in turn used to stress the strategies devised by humans. When it comes to deep uncertainties with a large number of hypotheses to explore, computing power is absolutely necessary to cope with the mass of data to be processed. On the other hand, the decision-making part and the creation of potential trade-offs then rely solely on the decision-makers who can use the results of the previous computational steps as support. This mix between computer science and human intelligence implies the use of a different set of tools. As the various applications of the RDM method show, it only defines the general type of tool you need for each step. The exact tools to use depend on the context and must be defined for each new project. A very quick overview and advice on the type of tools to use are detailed in the "step by step" section.

	[[File:RDM.drawio.png]]		[[File:RDM.drawio.png]]
Line 117:		Line 117:
	==Limitations ==		==Limitations ==

	As ~~presented previously~~, one of the ~~particularities~~ of RDM is that it is well ~~adapted to~~ situations with a high ~~level~~ of uncertainty. But the relevance of ~~using~~ this method and not the classical probabilistic risk analysis is really limited when we go ~~outside~~ this context. <ref name="open knowledge"> Robert Lempert, Nidhi Kalra, Suzanne Peyraud, Zhimin Mao, Sinh Bach Tan, Dean Cira, Alexander Lotsch (2013) Ensuring Robust Flood Risk Management in Ho Chi Minh City https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref>. Indeed, two main points are often mentioned in the literature:		As mentioned above, one of the peculiarities of RDM is that it is well suited for situations with a high degree of uncertainty. But the relevance of this method and not of the classical probabilistic risk analysis is really limited when we go beyond this context. <ref name="open knowledge"> Robert Lempert, Nidhi Kalra, Suzanne Peyraud, Zhimin Mao, Sinh Bach Tan, Dean Cira, Alexander Lotsch (2013) Ensuring Robust Flood Risk Management in Ho Chi Minh City https://openknowledge.worldbank.org/bitstream/handle/10986/15603/WPS6465.pdf?sequence=1&isAllowed=y </ref>. Indeed, two main points are often mentioned in the literature:

	- RDM requires high computing power because generating and exploring many possible futures has a high computational cost. <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. Thus, if the uncertainties are not deep and can be handled by the probabilistic approach, RDM will use unnecessary computing resources.		- RDM requires high computing power because generating and exploring many possible futures has a high computational cost. <ref name="name">https://doi.org/10.1007/978-3-030-05252-2 </ref>. Thus, if the uncertainties are not deep and can be handled by the probabilistic approach, RDM will use unnecessary computing resources.