Emergency Response Plan Validation

Deepwater Horizon - a topical Emergency Response Plan

Companies spend tens of thousands of dollars developing Emergency Response Plans...how do you know if it will work or not?  Most times you follow best practices, cross your fingers and hope.

As they said in the Senate hearings a few years back "Hope is not a strategy"

Is there a way to quickly and easily validate your Emergency Response Plan?  How would you go about  determining, bad luck and human frailties aside, that your plan is reasonable and stands a good chance of working?  Let's look at the options we would have a few years ago:

• Hire an expert or someone independent to look over your plan.  I guess that is better than nothing, since two pair of eyes are better than one, but that person may miss some key things.  Let's examine some facts.  Humans are poor judges of timing and sequence, so it is likely that critical problems will remain around time and space 
• Wargame the plan.  Gather a group of people together, each taking on a role (like a mini-tabletop exercise) and try and play the plan out bit by bit to see how it may play out.  The problem is that although this does effectively play the human interactions, once again the time and space realist is lacking.

How important is time and space?  Absolutely critical for an emergency!  So is there another tool that can help to validate the plan accurately and realistically?  Fortunately in the last few years, advances in simulation technology have allowed constructive simulations to play a part in plan validation.

First of all "garbage in, garbage out".  If you don't have a simulation that realistically plays time and space, you are wasting your time.  Secondly, if the simulation is not set up to accurately portray what you need simulated then it is not going to help you.  Finally, unless you have a simulation that you can easily change the scenario and rerun the event to ensure the changes make a positive impact within your plan, you are going to have a hard time determining exactly what is broken with your plan and how to fix it.

Step 1 - Select a simulation.

Choose a simulation tool that is easy for you to use, doesn't cost a lot of money to set up and operate and can be easily changed to exercise all aspects of your Emergency Response Plan.  Remember that you are going to have run run the simulation a bunch of times for each possible scenario to ensure that your plan will work, no matter what the circumstances.  If your constructive simulation tool is too hard to use or too hard to adjust, use a different one.

Step 2 – Plan your validation event

Determine what parts of your plan you are going to validate.  Determine how the emergency will unfold and ensure that you know how the emergency response is supposed to occur. 

Take the time to go through the data with a fine tooth comb:

• Is the mutual aid agreement really in place?  
• What resources are actually there, right now to deal with an emergency?  
• Is the equipment the same as listed in the plan?  
• Are the roads still the same?  
• What has changed?

In other words, make sure that the simulation you are going to run is realistic and accurately depicts the personnel, equipment and conditions that are really in place.  If pieces are missing or if you are unsure if they can respond, then exclude them from the scenario – you want to have an accurate to borderline worst case scenario, since you seldom read about disasters that unfurled in "best case" conditions.

How many simulation runs are you going to need?  You may need to run the simulation again with a different wind direction, or perhaps run it more than once under different weather conditions to ensure that your plan will work any time of the day or night, or any season.  Plan the scenarios carefully and only combine them if you are certain you will be able discern if one thing or another causes problems.  You don’t want to be left trying to figure out what went wrong – was it the weather or the time of day?

If it is important that your results be statistically valid, you need to decide how many times to run each scenario.  In some cases, multiple runs are needed in order to ensure that the data you collect is valid and any one-off aberrations are smoothed out.   

Step 3 – Run the simulation

Set up your simulation scenarios and run them.  During the simulation run you should follow along with the Emergency Response Plan to make sure that the simulation is doing what it is supposed to and that the right resources are responding at the right times.  Keep notes and record any hunches you may have about how the plan could be run more effectively.  At the end of the simulation run, record the Post Exercise Review information with a useful file name that is easy to reference to the original scenario.

Some constructive simulation tools allow you to play the scenarios in faster than real-time.  If you are doing this, be aware that you may miss critical events if the time is set too fast.  You may decide to slow down the simulation time during these critical events and then speed it up afterwards.

By the end of your simulation runs you should have a number of post exercise review files, copious notes and hunches that need verifying.  Don’t change anything yet!  Finish your experiments then analyze the data before making any changes to your plans or the scenarios.

Step 4 –Analyze the collected information

Carefully review your post exercise review files.  Just like a football coach reviewing the video of a game, you should be able to spot important things that you missed the first time around.  It is best to have a very inquisitive mind during these replays.  Openly wonder why something occurred that way and make a note of the question.

Go through your hunches and try to determine if any of the data support your hunch.  Think of a way to alter the scenario to test your hunch and prove it right or wrong.  “If I re-do it this way, I should be able to easily see if the roadblock needs to be set up further West”, and so on.

There is another important thing you should achieve here – Face Value Validation.  You can determine that your plan works in most circumstances and you should know where it will fail.  A successful plan should be able to easily handle the projected scenario but also be able to withstand the worst case scenario.

Keep a copy of your data and put it somewhere separate if you are running additional scenarios – you don’t want to contaminate your first runs with subsequent runs.  Then make the changes to the simulation scenarios and run the scenarios you need to run again.  You should now be able to prove or disprove your hypothesis – or maybe decide that you need to run a different set of scenarios because there was no clear outcome.

Step 5 – Incorporate the changes in the Emergency Response Plan

Once you have determined through your analysis of the simulation that a change is warranted in the Emergency Response Plan, take some extra time to think about it before you make a change.

• Was this clearly a time and space issue that had been overlooked in the original plan that the simulation has proven without a doubt…or do you need more research?
• Can you prove through another means that this change needs to happen?  Remember the “garbage in, garbage out”.  There may have been problems with the way the simulation was set up, so this is definitely a case of “measure twice, cut once”.

SUMMARY

Wow, it seems like a lot of work to use a constructive simulation to validate an Emergency Response Plan!  

Not really, actually.  If you were going to do the validation in real life, what you accomplish in a day with simulation would take months to do and would cost thousands of dollars (and think of the time planning and coordinating the activity!)

A good constructive simulation allows you to make moderate changes to the simulation scenario in minutes, so you are really doing one scenario and making several minor “tweaks” in how it is run.  The hard part is being organized with your data and being clinical with your analysis.  A good simulation will show you exactly where the major holes are in your Emergency Response Plan and will help you to craft the corrective changes you need to make it work properly.

IMPORTANT – even if you don’t have time to completely clinical validation of your Emergency Response Plan, at least run an exercise with it using a constructive simulation.  You will achieve the following benefits by running an exercise with your plan:

• Participants will know the plan better because they will be using it
• Participants will know better how to work together
• The plan will be exercised in at least one realistic setting, and that is better than doing nothing at all!

Finally, if this seems overwhelming, don’t be discouraged – it is not as hard to do as it seems.  Contact me and I can help you or point you to some resources who can assist.

Keep training!

Bruce