Using systems evaluation theory to improve points of dispensing planning, training, and evaluation

Ralph Renger, PhD, MEP; Brenda Granillo, MS, MEP

doi:10.5055/jem.2018.0364

Authors

Ralph Renger, PhD, MEP
Brenda Granillo, MS, MEP

DOI:

https://doi.org/10.5055/jem.2018.0364

Keywords:

evaluation, points of dispensing, Systems

Abstract

Recurring throughput problems served as a catalyst to search for a better approach to planning, training, and evaluating points of dispensing (POD) exercises. The paper begins with a discussion of the potential of systems thinking and systems theory to improve exercise planning, training, and evaluation. The paper then illustrates how systems concepts guided a POD design team in improving their planning, training, and evaluation strategy. The paper concludes by suggesting the application of systems thinking and systems theory to other emergency response strategies (eg, incident command, emergency operating centers) shows great promise, warranting further investment in testing its efficacy.

Author Biographies

Ralph Renger, PhD, MEP

University of North Dakota, Grand Forks, North Dakota

Brenda Granillo, MS, MEP

University of Arizona, Tucson, Arizona

References

National Preparedness Goal (NPG). Presidential Policy Directive 8: National Preparedness Goal. Second Edition. 2015. Available at https://www.fema.gov/media-library-data/1443799615171-2aae90be55041740f97e8532fc680d40/National_Preparedness_Goal_2nd_Edition.pdf.

Homeland Security Exercise and Evaluation Program (HSEEP): Homeland Security Exercise and Evaluation Program (HSEEP), 2013. Available at http://www.fema.gov/media-library-data/20130726-1914-25045-8890/hseep_apr13_.pdf.

Public Health Preparedness Capabilities: National Standards for State and Local Planning. Centers for Disease Control and Prevention. 2011. Available at https://www.cdc.gov/phpr/readiness/00_docs/DSLR_capabilities_July.pdf.

Annelli JF: The national incident management system: A multiagency approach to emergency response in the United States of America. Rev Sci Tech Off Int Epizoot. 2006; 25(1): 223.

Bogucki S, Schulz KJ: Incident Command System and National Incident Management System. Emergency Medical Services: Clinical Practice and Systems Oversight, Second Edition, pp. 255-263, 2015.

Jensen J, Waugh WL: The United States’ experience with the incident command system: What we think we know and what we need to know more about. JCCM. 2014; 22(1): 5-17.

Jensen J, Thompson S: The incident command system: A literature review. Disasters. 2016; 40(1): 158-182.

Ericson CA: Concise Encyclopedia of System Safety: Definition of Terms and Concepts. Hoboken, NJ: John Wiley & Sons, 2011.

FEMA: Integrated emergency management course. FEMA 208. Emmitsburg, MD: National Emergency Training Center. November, 1998.

Buckley WF: Sociology and Modern Systems Theory. Oxford, UK: Prentice-Hall, 1967.

Buckley WF: Society—A Complex Adaptive System: Essays in Social Theory. Amsterdam, the Netherlands: Gordon and Breach publishers, 1998.

Burns TR: System theories. In Ritzer G (ed.): The Wiley-Blackwell Encyclopedia of Sociology. Oxford, UK: Malden, MA and Blackwell Publishing, 2007.

Cohen AY, Kibel BM: The Basics of Open Systems Evaluation [Resource Paper]. Chapel Hill, NC: The Pacific Institute for Research and Evaluation, 1993.

Meadows DH, Wright D: Thinking in Systems: A Primer. White River Junction, VT: Chelsea Green Publishing, 2008.

Williams B, Hummelbrunner R. Systems Concepts in Action: A Practitioner’s Toolkit. Palo Alto, CA: Stanford University Press, 2010.

Adams KH, Hester PT, Bradley JM, et al.: Systems theory as the foundation for understanding systems. Syst Eng. 2014; 17(1): 112-123.

Checkland P: Systems Thinking. Rethinking Management Information Systems. New York, NY: Oxford University Press, 1999.

Von Bertalanffy L: General System Theory: Foundations, Development, Applications. New York, NY: George Braziller, 1968.

Williams JC: A systems thinking approach to analysis of the Patient Protection and Affordable Care Act. J Public Health Manage Pract. 2015; 21(1): 6-11. doi:10.1097/PHH.0000000000000150.

Renger R: System evaluation theory (SET). Eval J Australasia. 2015; 15(4): 16-28.

Renger R, McPherson M, Kontz-Bartels T, et al.: Process flow mapping for systems improvement: Lessons Learned. Can J Prog Eval. 2016; 31(1): 109-121.

Renger R: Illustrating the evaluation of system feedback mechanisms using system evaluation theory (SET). Eval J Australasia. 2016; 16(4): 15-21.

Renger R, Foltysova J, Ienuso S, et al.: Evaluating system cascading failures. Eval J Australasia. 2017; 17(2): 29-36.

Renger R, Foltysova J, Renger J, et al.: Defining systems to evaluate system efficiency and effectiveness. Eval J Australasia. 2017; 17(3): 4-13.

McCom SA, Green SG, Compton WD: Project goals, team performance, and shared understanding. Eng Manage J. 1999; 11(3): 7-12.

Green LW, Kreuter M: Health Program Planning: An Educational Approach. New York, NY: McGraw-Hill, 2005.

Santos JM Aguirre BE. Communicating risk and uncertainty: Science, technology, and disasters at the crossroads. In Rodriguez H, Diaz W, Santos JM, et al. Handbook of Disaster Research (pp. 476-488). New York, NY: Springer, 2007.

Toft B, Reynolds S: Learning from Disasters. Leicester, UK: Perpetuity Press, 2016.

Butler PW: Using leadership development programs to improve quality and efficiency in healthcare. J Healthcare Manage. 2008; 53(5): 319.

Boin A, Hart PT: Public leadership in times of crisis: Mission impossible? Pub Administr Rev. 2003; 63(5): 544-553.

Zaccaro SJ, Klimoski RJ (eds.): The Nature of Organizational Leadership: Understanding the Performance Imperatives Confronting Today’s Leaders (vol. 12). San Francisco, CA: Jossey-Bass, 2002.

Nelson E, Chan A, Chandra et al.: Recommended Infrastructure Standards for Mass Antibiotic Dispensing. Funded by the Department of Health and Humans Services Contract #DRR 14-43, 2008.

Huber G: Organizational information systems: Determinants of their performance and behavior. Manage Sci. 1982; 28(2): 138-155.

Renger R, Granillo B: Lessons learned in testing the feasibility of evaluating transfer of training to an operations setting. J Pub Health Manage Pract. 2014; 20: S30-S36.

CDC: Crisis emergency risk communication. U.S. Department of Health and Human Services, 1982. Available at https://emergency.cdc.gov/cerc/resources/pdf/cerc_2014edition.pdf 08/02/2017.

FEMA: IS 130: Exercise evaluation and improvement planning, 2017. Retrieved from https://emilms.fema.gov/IS130/indexMenu.htm 08-02-2017.

Stroud C, Viswanathan K, Powell T, et al.: Committee on Prepositioned Medical Countermeasures for the Public; Institute of Medicine. Prepositioning Antibiotics for Anthrax. Washington, DC: National Academies Press (US); Current Dispensing Strategies for Medical Countermeasures for Anthrax, 2011. Available at https://www.ncbi.nlm.nih.gov/books/NBK190045.