Stress tests

Introduction

Until the arrival of General Norman Schwarzkopf to United States Central Command (CENTCOM) in November 1988, American war planners assumed that the US’ biggest threat in the Middle East would come from a Soviet invasion of Iran. The Soviet Union had been a net exporter of oil until the late 1960s, but in order to meet its own growing industrial and energy demands Moscow looked increasingly to the Middle East and countries such as Iran to bolster its oil reserves.¹ Based on his own reading, however, Schwarzkopf turned this planning assumption on its head and with CENTCOM he worked on a wargame based on an intra-regional conflict between Kuwait, Iraq and Saudi Arabia rather than a Soviet invasion. Apart from the precise details about potential US fatalities (Schwarzkopf and his team believed the US would suffer more casualties than it actually did), the general and his wargamers were right: Iraq eventually invaded and annexed Kuwait in August 1990 and Operation Desert Storm was over before the dissolution of the Soviet Union in December 1991. Schwarzkopf’s wargame had been prophetic in preparing the US military for war with Iraq.²

Of course, the history of wargaming and scenario planning does not always mirror the success achieved by Schwarzkopf. Companies such as Shell that are famed for their in-house scenario planning departments do have notable periods of success, but the company’s ‘futures branch’ was nearly shut down three times in almost 50 years of scenario planning at the firm because Shell’s leaders saw little value in its work.³ In some cases, sound scenario planning and the assessments they lead to are not taken seriously enough: for example, vulnerability assessments of the Fukushima nuclear plant showed that there was a risk from natural disasters such as tsunamis but the consequences of these assessments were downplayed.⁴ Most of the time, however, crisis scenario planning unfortunately emerges in the wake of a major crisis such as the terrorist attacks of 11 September 2001 in the US or the global financial crisis of 2007.⁵

This is partially true for the EU as well. Even though the Union has organised Crisis Management Exercises (CMEs) under the Common Security and Defence Policy (CSDP) since May 2002⁶ and a host of other exercises in areas such as civil protection,⁷ the aftermath of Russia’s seizure of Crimea and the advent (or re-emergence) of ‘hybrid threats’ has intensified the type and frequency of EU crisis scenario planning. In September 2017, the EU held its first ministerial-level table top exercise on cyber defence (‘EU CYBRID17’) and in January 2018 the European Commission organised an exercise on public health and hybrid threats (‘Exercise Chimera’). What is more, in September 2017⁸ and November 2018 the Union organised Parallel and Coordinated Exercises (PACE) on a hybrid threat affecting the EU. In particular, the November 2018 exercise called ‘EU-HEX-ML 18’ saw the EU simulate a combined border, terrorist and cyber-related crisis in the fictitious countries of Ropperta and Loripa.⁹

One of the key features of the EU’s recent crisis exercises is the participation of multiple institutional actors. Bodies such as the Council of the EU and its Integrated Political Crisis Response (IPCR) mechanism, the crisis response mechanisms of the European External Action Service (EEAS) and Directorates General (DG) of the European Commission (e.g. DG ECHO, DG Home, DG Sante and more) respond to trans-boundary crises such as hybrid threats, terrorism and criminal activity, public health, energy security and more. In addition, the EU Institute for Security Studies (EUISS) has designed, planned and managed crisis scenarios.¹⁰ In two ‘What if…’ collections published respectively in 2017¹¹ and 2019,¹² the EUISS created 23 crisis scenarios ranging from the terrorist takeover of a cruise liner in the Mediterranean by Daesh, to a second ‘Arab Spring’, the hijacking of a foreign submarine on EU territory, a clash between Russia and Turkey in the Caucasus and more.

Despite the increased frequency of hybrid threat exercises and scenarios in the EU, and notwithstanding the rare success of individuals such as General Schwarzkopf in the past, there a number of questions associated with the design and practice of crisis scenarios that need probing. These include: what is the purpose or objective of crisis scenarios? What institutional or political change should crisis scenarios lead to? How should we go about designing and structuring scenarios? For the EU in particular, another question arises: how can crisis scenarios effectively bring together a plethora of crisis bodies and actors given the Union’s institutional complexities? Addressing such questions, this Brief is split into two main parts. Part one looks at the critique of crisis scenario planning and suggests ways in which simulations and exercises can be useful. Part two provides a ‘walkthrough’ of scenario planning, and, by utilising a fictitious crisis situation, delves further into some of the practical considerations associated with running crisis scenarios. This Brief should be seen as an analytical contribution to EU institutional and member state crisis planning, but it should be of use to think tankers, academics and private sector actors, too.

Averting or avoiding crises?

Scholars in the field of crisis management have long pointed to the challenges associated with scenario planning. Most of the time, however, this involves deflecting the cynical accusation that exercises or simulations are meaningless. Here, the criticism is that scenarios are a good way of side-stepping real-life security challenges and/or glossing over the dysfunction of an organisation charged with crisis response. The harshest critics state that while crisis scenarios may have symbolic value, they do not necessarily enhance operational readiness or crisis preparedness.¹³ Even some supporters of crisis scenarios agree that simulations and exercises are only really useful if they help overcome the notoriously fragmented nature of public, private and civil society approaches to crises.¹⁴ Another criticism often levelled at scenarios is that they rarely live up to the unpredictable nature of crises, which very rarely proceed in an orderly manner and cannot usually be managed by pre-packaged plans.¹⁵ Furthermore, political leaders or business chiefs do not always see why they should invest scarce resources into scenario planning for low-probability events.

Part of this criticism is rooted in a misunderstanding of what crisis simulations are all about; too often, scenarios are associated with ‘prediction’. This is not true. Instead, rather than prediction scenarios rely on forecasting methods such as an analysis of data trends over a defined period of time. Compared with prediction, forecasting in fact allows academics to test and build theories while policymakers can use forecasting to uncover a range of variables that may affect their response mechanisms in times of crisis.¹⁶ For example, when Shell planned for future crises in the 1970s they developed a range of foresight studies on what might happen to supply inventories and whether politics in the Middle East would lead to oil nationalisation. It was only once Shell had completed their foresight studies that they were able to draw up a number of crisis scenarios about what the company would do if any of the futures materialised. We know with hindsight that Shell was able to use both foresighting and simulations to cushion the blow of the 1973 oil crisis by foreseeing oil nationalisation and surmising that the price per barrel would rise from $2 per barrel to $10 by 1975 (the actual price in 1975 was $13 per barrel).¹⁷ These crisis scenarios led the firm to shore up oil inventories and so avert a major shock.

Foresight studies and crisis scenarios should reinforce each other. Ideally, crisis scenarios should be preceded by foresight work and combined with staff training, vulnerability and risk assessments, crisis coordination mechanisms, strategic communication, resource generation and financial resources.¹⁸ If all of these elements are in place, engaging regularly with fictitious scenarios – regardless of how unrealistic they may be – can broaden the horizon or perceptions of policymakers by taking them out of their comfort zone and encouraging them to think beyond the present moment. It is because ‘almost every crisis contains within itself the seeds of success as well as the roots of failure’ that it is risky to neglect the learning opportunities associated with creating and walking through crisis scenarios.¹⁹ What is required is for each crisis scenario to have a set of informed forecasts at their heart rather than ad hoc predictions.

Thus, crisis simulations should not be dismissed so easily, not least because engaging in exercises and simulations are a form of strategic communication that shows that an organisation is prepared (or preparing) for conventional and unconventional threats. Through exercises and simulations, an organisation or body can communicate to partners and adversaries how it perceives and responds to threats. This is certainly the case with military exercises, regardless of whether adversaries and partners are invited as official observers or not. For example, the Cuban Missile Crisis of 1962 gave way to one of the US military’s largest wargames called the ‘Simulation of Total Atomic Global Exchange’ in which computer models and military planners confirmed that the US could survive an atomic war. Unsurprisingly, the US did not keep this conclusion secret from the Soviet Union despite the highly secretive nature of the wargame. The same was true of an inter-agency war game held by the US in March 1982, where a normally secretive wargame on nuclear strikes was made public by the Reagan presidency to highlight the US government’s ability to provide continuity after a Soviet strike.²⁰ The danger, of course, is that political leaders see in exercises a communicative tool with no effort to invest in real response capabilities or reform crisis response mechanisms.

What is required is for each crisis scenario to have a set of informed forecasts at their heart rather than ad hoc predictions.

Simulations and exercises are also an opportunity to exchange information and produce shared understandings of crisis response. The ancient Greeks ­appreciated the educational aspects of simulations, as demonstrated by their use of board games (kuboi and pessoi) in order to pass on knowledge from military commanders to troops and cadets.²¹ Even today, simulations and exercises provide the police, military, health professionals, engineers, government, firms, etc. with a space to verify their own crisis response plans and capabilities. This space does not automatically contribute to unity of purpose because organisations may use it to forward their interests or engage in a vanity contest to promote individual approaches or mechanisms. In theory, however, should participants genuinely commit to collectively improving crisis response then these spaces may reveal institutional, psychological and normative biases and assumptions or other weak points. This sort of learning exchange is vital if the costs of crises are to be potentially minimised. In the EU, this is of increasing importance given the range of institutions and bodies involved in crisis response. The EU crisis architecture is spread over a range of bodies that encapsulate different mindsets and priorities. For example, DGs in the Commission have a civilian approach to crisis response which is different to the military approach taken by the EU Military Staff (EUMS) and EU Military Committee (EUMC). Accordingly, simulations and exercises can be a way of helping planners and officials question silo mentalities before a real crisis emerges.

Additionally, given that each EU member state locates their respective crisis response centre(s) in different ministries and agencies, organising a crisis scenario at the EU level is a good way of ‘mapping out’ these centres and establishing connections between member states and institutions. While the EU has traditionally centred its CMEs on better civil-military coordination under the CSDP,²² today there is a need to think about a more all-encompassing definition of crisis response that includes chemical, biological, radiological and nuclear (CBRN), disinformation, the protection of critical infrastructure, cyber-attacks, border management. As the EU’s 2014 ‘exercise policy’ under the Common Foreign and Security Policy (CFSP) makes clear, Union exercises should address emerging crises; time-sensitive crisis; technological incidents; man-made or natural disasters; risks and threats to EU space systems (e.g. Galileo and Copernicus) and asymmetric threats.²³

^{Timelines of select crises}

Unleashing ‘Virus Z’…

Even if crisis scenarios are worth the investment of time and resources, there are a number of challenges associated with designing them. To start with, a scenario planner should not jump straight into thinking about geomagnetic storms, a plane hijacking, alien invasions or a Third World War. Before the creative part of scenario planning gets underway, it is critical to define objectives. Such objectives can include stress testing the response time of crisis responders, the institutional coherence of the response and the capability inventories of crisis management teams. A scenario can test each of these separately or together.

If the objective of the crisis simulation or exercise is to test the response times of emergency services and political authorities, then the scenario has to be designed in such a way as to create realistic crisis timelines. Individual crisis situations occur over different time frames. For example, during the Fukushima nuclear disaster of 2011 it took approximately 45 minutes between the initial earthquake near Honshu Island and the subsequent tsunami to strike the Fukushima Daiichi reactor. A further 30 minutes were required for the Nuclear and Industrial Safety Agency of Japan to establish a crisis headquarters.²⁴ Accordingly, a simulation based on a similar nuclear disaster should not last more than two hours if the objective is to test the initial response time of crisis mechanisms. Likewise, if the objective is to stress test how an organisation or government would respond to a pandemic then a full day’s simulation is needed to account for the fact that viruses can take anywhere between a few months and years to spread and lead to fatalities.

If institutional coherence or the generation of capability inventories are the key objectives, however, then it should be determined which organisation/s or actors this scenario would be most applicable to. Imagine, for example, that China begins dredging work to create an artificial island somewhere in the high north of Europe (e.g. near Jan Mayen island between Greenland and Norway) to militarily support its access to trade openings in the Arctic Ocean. If the task is to improve institutional coherence then when participants are brought together to address a particular crisis the focus should not be on solving the crisis situation per se, but rather walking through the response and mapping the political chain of command. First and foremost, because a Chinese artificial island in the high north of Europe is likely to elicit a response at the United Nations it is perhaps not a scenario that would necessarily trigger traditional crisis response mechanisms such as aid relief, civil protection, etc.

Accordingly, if we wanted to stress test all three objectives in an EU context then a cross-border scenario is perhaps better suited. We could, for example, design a scenario whereby a light aircraft crashes somewhere in the EU and a few days later an unknown contagious virus (let us call it ‘Virus Z’) emerges near the crash site with increasing numbers of people hospitalised as a result. Such a scenario gives us two avenues to pursue simultaneously. First, a pandemic is a good way to mobilise an EU-wide response because it has a trans-boundary element. By not specifying what virus it is, not only do we add an extra layer of urgency for participants but we can control the time period in which the scenario takes place. For example, the 2009 ‘(H1N1)pdm09’ influenza pandemic led to 274,304 hospitalisations in the space of a single year.²⁵ Here, we would also have to decide if the virus is transmitted only by human-to-human contact or whether it is also zoonotic and spread by animals such as migratory birds. Second, the fact that the virus is related to a plane crash in the same vicinity opens up the possibility that there was malicious intent behind the outbreak of the virus. For example, a terrorist group or hostile foreign power could have deliberately crashed the plane with infected people on board and thereby exposing the emergency responders to the virus.

Given the political sensitivities that may surround this scenario, it is up to the simulation planner to decide whether to use real countries and actors or to use fictitious ones. Using real actors and countries has the benefit of enhancing the experience for participants because non-fiction focuses minds and emotions,²⁶ but using non-fictitious actors means that the scenario planner does not have to spend too much time inventing names and actors. It is important to note that participants will have to expend more time and intellectual energy on learning about fictitious entities than is the case with real actors and countries. Affording participants the time to learn about fictitious characters and countries is time they could be spending on the scenario itself. Of course, non-fictitious scenarios are best played behind closed doors, especially if the characters or countries used are a cause for concern in real life and there may be sensitive or classified aspects about the content used and shared.

For our purposes, we will assume that the light aircraft landed in a northern European country called Vallaland and that the craft bore the insignia of a non-EU country called Surik that is a security concern for the Union. For our scenario to work, it is best that Virus Z emerges in a geographical area that will allow for it to spread across borders (i.e. island states would not be particularly useful in this regard).

Once the basis of a crisis scenario is in place, the next step is to think about whether we want to play the scenario mainly at the strategic or the tactical level. Military exercises, for example, are usually played at the tactical level and this means thinking about capabilities and crisis response entry points. Following the scenario we have set, a tactical approach would focus on how Vallaland would deal with the crash site and whether transport nodes should be closed to stop the spread of the Virus Z into the neighbouring countries of Surima and Gorgulu. A tactical approach may also involve thinking about the capacity of hospitals in Vallaland and how national authorities could communicate their response to the general public and neighbours. By way of a response, this more tactical approach may seek to limit the spread of Virus Z within Vallaland and thereby stress test its national crisis response capacities. This is a beneficial aspect of thinking about tactical decisions because it helps us game which specific government ministries, agencies and services are involved in crisis response and what national response ‘playbooks’ look like.

'Virus Z' scenario map

Data: GADM, 2019; Natural Earth, 2019

In addition to this tactical focus, it is worth introducing elements in the scenario that will give way to strategic or more political responses. Remember, we are trying in this instance to test the EU’s crisis response architecture. Therefore, let us picture a situation in our scenario where neighbouring countries Surima and Gorgulu report cases of Virus Z. Let us then assume that after escalating the situation to the national politico-strategic level and recognising its limited options, the government of Vallaland takes the decision to invoke the EU’s ‘solidarity clause’ (Article 222 TFEU), thereby triggering an EU-wide response steered in the Council of the EU and bringing together the Commission and High Representative services, with the support of the IPCR arrangements. This would therefore also involve meetings of the Crisis Coordination Committee and the initiation of the Commission’s internal crisis coordination system ARGUS. Although CSDP capacities would not be applicable to the scenario we have created, the EEAS Crisis Response Mechanism could nevertheless be used to deal with the hybrid threat element of the scenario (i.e. whether or not Surik was behind the deliberate crash of the light aircraft).

Evolving the scenario in this way would be particularly helpful in bringing together diverse EU institutional actors. It would allow crisis responders to develop networks that can better share knowledge, and, at a more basic level, even telephone numbers and email addresses. Do not, however, underestimate the difficulties of bringing together these actors. In fact, one of the many conclusions that may emerge from our scenario is that it may be difficult to identify who takes the lead in responding to Virus Z (in terms of treatment and slowing contagion) and who deals with the hybrid threat aspect (in terms of intelligence gathering, possible attribution and punitive measures). When the Solidarity Clause is invoked it is relatively simple to identify who should take the lead – i.e. the Council of the EU ‘shall ensure the political and strategic direction of the Union response’.²⁷ Outside of Article 222 situations, however, the picture becomes more complex: ultimately, authorities in Vallaland, Surima and Gorgulu may want to take the lead but EU institutions could be involved in coordinating Union-wide efforts. This is why a scenario session of this kind should focus on walking through political responsibility and EU coordination mechanisms.

Of course, non-fictitious scenarios are best played behind closed doors.

What may also likely emerge during any crisis simulation is the role of information and/or intelligence. For example, we do not know if Vallaland, Surima or Gorgulu would be prepared to share their intelligence assessments of whether the plane crash and the outbreak of Virus Z are connected. These countries may also not be prepared to share information with the EU institutions, either because they do not actually have any credible intelligence, which would raise questions about their national preparedness, or because they want to politically control the crisis response by maintaining an information asymmetry between their capital and the EU. To probe such factors, such a scenario would have national and EU crisis planners present so that it is clear how national crisis response mechanisms connect (or not) with the EU crisis architecture. Of course, the inevitable risk with this theoretical assumption is that national authorities will be more eager to share information with each other and supporting EU actors during an exercise than they would be in a real life crisis. Information is central to any scenario and, like all non-fictitious crises, a lack of reliable information is to be expected. For the purposes of our scenario, it would not be necessary to provide information on every aspect of the crisis: for example, there is no need to describe Virus Z in too much detail because the primary concern is not to create a vaccine for the virus in the early stages of the crisis. Much like real life, the element of surprise is useful in shocking participants into action, although we should not starve participants of the basic information they would have in real life so as to avoid disengagement.²⁸

Crisis scenario planners have a lot of influence over the direction in which a simulation heads next. We could, for example, simply end the scenario with responses like the closure of airports in Vallaland, Surima and Gorgulu or the provision of quarantine equipment to the affected member states. However, we may also want to escalate the crisis – even though an excessively complex (or acute) simulation may result in a loss of attention or interest by participants. Nevertheless, let us add a few more rogue elements into our scenario. For example, what if envelopes containing spoiled tissue paper and messages stating that the contents are infected with Virus Z start turning up at hospitals, government buildings and schools in other parts of the EU. This evolution would immediately force us to think about whether the member states’ response and the EU’s supporting crisis mechanisms that have been triggered thus far need augmenting or bolstering. From a hybrid threat perspective, it would also mean a greater focus on whether a non-EU state or terrorist group is behind the posted envelopes and whether or not there is a link between this new threat and the initial aircraft crash. In any case, this additional stage in the crisis may force us to re-think about the crisis response interface between EU member states and the institutions.

Response typologies

A means to an end…

It should go without saying that the venue in which one holds a crisis simulation is important, and the techniques one can use to encourage participation are plentiful. However, we do not necessarily have to resort to ‘dirty’ diplomatic negotiation strategies such as putting participants in a small room, removing their smart phones or depriving them of sleep until a conclusion is arrived at.²⁹ Instead, one of the most important objectives and conclusions of any crisis simulation is networking and generating a shared understanding of crisis response. In fact, one of the benefits of scenarios is that they often initiate a learning process that should take root after the simulation has ended. In an inter-organisational or intra-institutional setting, the aim is to ensure that a critical mass of individuals want to carry forward the conclusions of a simulation or exercise – either to change behaviours and psychologies within their organisation or to improve the overall crisis response architecture. The whole point about simulations and exercises is that they lead to new knowledge. In this sense, it is vitally important to focus on both the successes and (perhaps even more so) the failures observed during a crisis simulation.³⁰

It cannot be stressed enough that conducting simulations or exercises in isolation from broader efforts to enhance institutional coherence and crisis response capacities is an error.

Finally, it cannot be stressed enough that conducting simulations or exercises in isolation from broader efforts to enhance institutional coherence and crisis response capacities is an error.³¹ Let us return to where we began this Brief with the example of General Schwarzkopf: his wargame alone did not prepare the US military for war against Iraq. The ideal situation is to organise crisis simulations and exercises alongside the development of institutional coherence and crisis response capacities. Yet, ultimately, the added-value of crisis scenarios rests on having political buy-in from leaders and senior officials. Without having a hierarchy that understands that simulations and exercises form part of wider crisis preparedness efforts, then it is questionable whether crisis scenario events will lead to a shift in an institution’s approach to crises. Fortunately for the EU, exercises and simulations are a feature of crisis preparedness even if more can be done to enhance the crisis response linkages between the EU member states and Brussels-based institutions.

References

1) John A. Berry, “Oil and Soviet Policy in the Middle East”, The Middle East Journal, vol. 26, no. 2 (1972), pp. 149-160.

2) See Anthony H. Cordesman, Lessons of the Gulf War: 1990-1991 (Washington, DC: Center for Strategic and International Studies, 2013) p. 42 and Matthew B. Caffrey Jr., “On Wargaming : How Wargames have Shaped History and How they may Shape the Future”, Newport Papers, no. 43, US Naval War College, 2019, p. 130.

3) Angela Wilkinson and Roland Kupers, “Living in the Futures”, Harvard Business Review, May, 2013, https://hbr.org/2013/05/living-in-the-futures.

4) James M. Acton and Mark Hibbs, “Why Fukushima was Preventable”, The Carnegie Papers, March 2012, Carnegie Endowment for International Peace, https://carnegieendowment.org/files/fukushima.pdf, p. 25.

5) Barbara Bilodeau and Darrell K. Rigby, “A Growing Focus on Preparedness”, Harvard Business Review, June-August, 2007, https://hbr.org/2007/07/a-growing-focus-on-preparedness.

6) The first known CME conducted under the CSDP (or European Security and Defence Policy (ESDP), as CSDP was known before the Lisbon Treaty) took place between 22-28 May 2002. It is noteworthy that the Western European Union also regularly exercised with NATO during the 1990s and early 2000s. The EU’s CMEs emerged after the ESDP was created in 1999 with the Treaty of Amsterdam.

7) For example, there have been EU civil protection and disaster management exercises such as the European Disaster Response Exercise (EDREX), EU Prometheus and ARETE. A list of example civil protection exercises can be be consulted here: https://ec.europa.eu/echo/funding-evaluations/financing-civil-protection/civil-protection-exercises_en.

8) European External Action Service, “EU Launches Exercise to Test Crisis Management Mechanisms in Response to Cyber and Hybrid Threats”, September 28, 2017, https://eeas.europa.eu/headquarters/headquarters-homepage/32969/eu-launches-exercise-test-crisis-management-mechanisms-response-cyber-and-hybrid-threats_en.

9) Council of the EU, “EU HEX-ML 18 (PACE) EU Hybrid Exercise Multilayer 18 (Parallel and Coordinated Exercise) Exercise Instructions (EXINST)”, 13577/18, Brussels, October 26, 2018, https://data.consilium.europa.eu/doc/document/ST-13577-2018-INIT/en/pdf.

10) See, for example, the events that the EUISS organised on 28 February in Bucharest https://www.iss.europa.eu/content/facing-hybrid-threats-through-consolidated-resilience-and-enhanced-strategic-communication and 4 April in Brussels https://www.iss.europa.eu/content/sense-making-decision-making-sharing-experiences-management-transboundary-crises-eu.

11) Florence Gaub (ed.) “What if… Scanning the Horizon: 12 Scenarios for 2021”, EUISS Chaillot Paper, no. 150, 2019, https://www.iss.europa.eu/content/what-if-scanning-horizon-12-scenarios-2021.

12) Florence Gaub (ed.) “What if… Conceivable Crises: Unpredictable in 2017, Unmanageable in 2020?”, EUISS Report, no. 34, June, 2017, https://www.iss.europa.eu/content/what-if-conceivable-crises-unpredictable-2017-unmanageable-2020.

13) Allan McConnell and Lynn Drennan, “Mission Impossible? Planning and Preparing for Crisis”, Journal of Contingencies and Crisis Management, vol. 14, no. 2 (2006), pp. 59-70.

14) Ibid.

15) See for example, Carl Boettiger and Alan Hastings, “From Patterns to Predictions”, Nature, no. 493, January 10, 2013, pp. 157-158.

16) Havard Hegre et al., “Introduction: Forecasting in Peace Research”, Journal of Peace Research, vol. 54, no. 2 (2017), pp. 116-117.

17) Michael Jefferson, “Shell Scenarios: What Really Happened in the 1970s and What May Be Learned for Current World Prospects”, Technological Forecasting & Social Change, vol. 79 (2012), pp. 186-197.

18) World Health Organisation, “A Strategic Framework for Emergency Preparedness”, Geneva, 2016, https://extranet.who.int/sph/sites/default/files/document-library/document/Preparedness-9789241511827-eng.pdf.

19) Norman R. Augustine, “Managing the Crisis You Tried to Prevent”, Harvard Business Review, November-December, 1995, https://hbr.org/1995/11/managing-the-crisis-you-tried-to-prevent.

20) Op.Cit., “On Wargaming”, p. 80 and p. 90.

21) Leslie Kurke, “Ancient Greek Board Games and How to Play Them”, Classical Philology, vol. 94, no. 3 (1999), p. 250.

22) Council of the EU, “Draft EU Training Concept in ESDP”, 11970/04, Brussels, August 30, 2004, http://www.eeas.europa.eu/archives/docs/csdp/documents/pdf/trainingconcept_en.pdf.

23) Council of the EU, “Exercise Policy of the European Union under the Common Foreign and Security Policy (CFSP)”, 18047/1/13, Brussels, April 1, 2014, p. 4.

24) International Atomic Energy Agency, “The Fukushima Daiichi Accident”, Technical Volume 1/5 – Description and Context of the Accident, 2015, https://www-pub.iaea.org/MTCD/Publications/PDF/AdditionalVolumes/P1710/Pub1710-TV1-Web.pdf.

25) Centers for Disease Control and Prevention, “2009 H1N1 Pandemic (H1N1pdm09 virus)”, June 11, 2019, https://www.cdc.gov/flu/pandemic-resources/2009-h1n1-pandemic.html.

26) Paul ‘t Hart, “Preparing Policy Makers for Crisis Management: The Role of Simulations”, Journal of Contingencies and Crisis Management, vol. 5, no. 4 (1994), p. 210.

27) “Council Decision on the Arrangements for the Implementation by the Union of the Solidarity Clause”, Decision 2014/415/EU, June 24, 2014.

28) Tony Ingesson, “Anticipating the Zombie Apocalypse: Using Improbability to Teach Intelligence Analysis”, International Journal of Intelligence and Counter-Intelligence, vol. 32, no. 1 (2019), p. 381.

29) Paul Meerts, Diplomatic Negotiation: Essence and Evolution (The Hague: Clingendael Institute, 2015), p. 54.

30) Op.Cit., “Preparing Policy Makers for Crisis Management: The Role of Simulations”, p. 215.

31) Arjen Boin, Celesta Kofman-Bos and Werner Overdijk, “Crisis Simulations: Exploring Tomorrow’s Vulnerabilities and Threats”, Simulation & Gaming, vol. 35, no. 3 (2004), pp. 378-393.