Open Access Open Access  Restricted Access Subscription or Fee Access

Using evaluation theory to augment the homeland security exercise and evaluation program (HSEEP) guidance for evaluating operations-based exercises

Ralph Renger, PhD, MEP, Anneke Jansen, MPH, Erin Peacock, MPH, Adriana Cimetta, MPH, Jessica Surdam, MPH

Abstract


Exercises play a crucial role in better preparing for, responding to, and recovering from an emergency by providing opportunities for responders and officials to practice and assess their collective capabilities. Conducting a thorough evaluation of these exercises is critical to ensuring that the nation continually improves its ability to save lives and property. A major emphasis of the Homeland Security Exercise and Evaluation Program (HSEEP) is on defining and evaluating capability- based objectives to determine the impact of an exercise. Using the integrated theory of evaluation, it is shown how a cost-effective, quality evaluation of operations- based exercises can be conducted while simultaneously not interfering or adding to the burden of exercise players, controllers, or evaluators. It is hoped that this article will act as a catalyst in moving HSEEP to recognize the potential of other sources of information to assist in conducting a more comprehensive evaluation and amend their guidelines accordingly.

Keywords


evaluating, HSEEP, operations-based, exercises, theory

Full Text:

PDF

References


United States Office of Management and Budget [OMB]: Detailed information on the Federal Emergency Management Agency: Grants and Training Office National Exercise Program Assessment [Web page]. Available at http://www.whitehouse.gov/omb/expectmore/detail/10003606.2005.html. Accessed October 16, 2007.

Redlener I: Americans at Risk: Why We Are Not Prepared for Megadisasters and What We Can Do. New York, New York: Knopf Publishing Group, 2006.

Federal Emergency Management Agency: ICS-100: Introduction to ICS Student Manual. Emmitsburg, MD: Emergency Management Institute, 2005.

United States Department of Homeland Security: Homeland Security and Evaluation Program (HSEEP), Volume 1: HSEEP overview and exercise program management. Available at https://hseep.dhs.gov/support/VolumeI.pdf. Accessed May 1, 2007.

United States Department of Homeland Security: Homeland Security Exercise and Evaluation Program: Volume II exercise planning and conduct. Available at: https://hseep.dhs.gov/support/ VolumeII.pdf. Accessed May 1, 2007.

United States Department of Homeland Security: Homeland Security Exercise and Evaluation Program: Volume III exercise evaluation and improvement planning. Available at https://hseep.dhs.gov/support/VolumeIII.pdf. Accessed May 1, 2007.

Department of Homeland Security: Homeland Security Exercise and Evaluation Program, terminology, methodology, and compliance guidelines.Available at https://hseep.dhs.gov/support/HSEEP_101.pdf. Accessed February 7, 2008.

Federal Emergency Management Agency: IS-130: Exercise Evaluation and Improvement Planning. Emmitsburg, MD: Emergency Management Institute, 2008.

Federal Emergency Management Agency: Homeland security exercise and evaluation program terminology, methodology and compliance guidelines. Available at https://hseep.dhs.gov/support/HSEEP%20101.pdf. Accessed October 3, 2007.

Federal Emergency Management Agency: ICS-700: National Incident Management System (NIMS), An Introduction Self-Study Guide. Emmitsburg, MD: Emergency Management Institute, 2004.

United States Department of Homeland Security: National Incident Management System. Available at http://www.fema.gov/pdf/emergency/nims/nims_doc_full.pdf. Accessed May 1, 2007.

United State Department of Homeland Security: Target capabilities list a companion to the national preparedness guideline, 2007. Available at https://www.llis.dhs.gov/getFile.cfm?id=26724. Accessed October 16, 2007.

Mark MM, Henry GT, Julnes G. Evaluation: An Integrated Framework for Understanding, Guiding, and Improving Policies and Programs. San Francisco: Jossey Bass, 2000.

Office of Management and Budget: Government Performance Results Acts (GPRA). Washington, DC: The White House, 1993. Available at http://www.whitehouse.gov/omb/mgmt-gpra/gplaw2m.html. Accessed May 1, 2007.

Conrad E, Maul T. Introduction to Experimental Psychology. New York, NY:Wiley, 1981.

Joint Committee on Standards for Educational Evaluation. The Program Evaluation Standards: How to Assess Evaluations of Educational Programs, 2nd ed. Thousand Oaks, CA: Sage, 1994.

Renger R Consequences to Federal Programs when the logic modeling process is not followed with fidelity. Am J Eval. 2006; 27(4): 452-464.

Federal Emergency Management Agency. ICS Resource Center: ICS Forms [Web page]. Available at http://training.fema.gov/emiweb/icsresource/ICSResCntr_Forms.htm. Accessed May 1, 2007.




DOI: http://dx.doi.org/10.5055/jem.2008.0021

Refbacks

  • There are currently no refbacks.


Copyright (c) 2018 Journal of Emergency Management
This site uses cookies to maintain session information critical to the user's experience and environment on this system. Click "Accept Cookies" to continue.
For more details please visit our privacy statement at: Privacy & GDPR