EXERCISE EVALUATION AND FOLLOW-UP
largely a guided discussion, involving
organizers, designers, evaluators, observers
and participants, aimed at providing an
opportunity for all to express overall
impressions, concerns and views.
which ended up poorly. However, it is also
critical that the right people be present people with the power to move things forward.
Provide these people with the AAR to have
something to refer to.
Some of the basic principles for achieving an
objective assessment of the current
state/performance include a frank and open
assessment of actual conditions. This is
especially relevant in cases of poor
performance or failure, when leadership might
not be willing to accept anything less than good
news. Evaluation should always be considered
a way of pinpointing areas for/of improvement,
rather than an excuse for blaming or fingerpointing. The goal is to further sharpen
participant skills, broaden their experiences
and fill gaps identified during the exercise. So
as not to give the impression of blaming,
evaluations should always contain potential
solutions to the identified shortcomings.
The evaluation process and follow-up event
should lead to the creation of a best practices
document which constitutes an important part
of each exercise. Its purpose is to outline what
to do in case of such a crisis/scenario, to serve
as a step-by-step manual in dealing with
serious incidents. As the best practices
document is one of the most useful and
practical outcomes for the participants, make it
available to your partners and other
stakeholders. The more people and institutions
are familiar with this document, the more
educated and prepared the community it is.
Evaluation outcomes emerging from feedback
and debrief sessions should be further
elaborated in an After Action Report (AAR).
Take your time with preparing this summary
report (it usually takes several weeks). A highquality AAR describes the exercise, what the
main objectives were and if (and how) they
were met. Key aspect of the AAR include
highlighting the major weak spots and giving
recommendations on how to minimize or
eliminate shortcomings observed in the
exercise. Without drafting solutions and
countermeasures, the AAR (and even the
whole exercise) loses its potential to serve as a
key supporting tool in promoting the remedy of
insufficiencies and driving organizational
change (at management and/or political
levels).
Another means for following up on emergent
issues is to organize a follow-up event, invite
respective decision-makers, top management
and political leaders, detail key weak spots and
jointly discuss possible solutions. It is always
better then if the executives have gone
through the scenario and experienced the
virtual negative consequences of a scenario
Finally, the essential implementation stage
remains to be addressed. Once you have
identified weak spots and have produced an
AAR and best practice summary, it is time to
implement all of the findings into real life
wherever feasible. It is likely that cyber
exercises will uncover insufficiencies in current
procedures, process, SOPs and policies
triggered in the wake of a cyber incident. Lay
out the standard operating procedures,
business continuity plans, disaster recovery
plans on the table, assess their adequacy and
revise them to avoid the same shortcomings.
Develop a plan B for all possible scenarios. Do
not neglect to educate senior leadership and
other applicable audiences regarding the
possible risks and threats.
What to exercise next time? Test and validate
whether all revised plans and procedures
would suffice and work in a cyber crisis. Find
out what progress has been done in this area.
26/29