Figures
Abstract
Objective
To describe an innovative approach for developing and implementing an in-service curriculum in China for staff of the newly established health emergency response offices (HEROs), and that is generalisable to other settings.
Methods
The multi-method training needs assessment included reviews of the competency domains needed to implement the International Health Regulations (2005) as well as China’s policies and emergency regulations. The review, iterative interviews and workshops with experts in government, academia, the military, and with HERO staff were reviewed critically by an expert technical advisory panel.
Findings
Over 1600 participants contributed to curriculum development. Of the 18 competency domains identified as essential for HERO staff, nine were developed into priority in-service training modules to be conducted over 2.5 weeks. Experts from academia and experienced practitioners prepared and delivered each module through lectures followed by interactive problem-solving exercises and desktop simulations to help trainees apply, experiment with, and consolidate newly acquired knowledge and skills.
Conclusion
This study adds to the emerging literature on China’s enduring efforts to strengthen its emergency response capabilities since the outbreak of SARS in 2003. The multi-method approach to curriculum development in partnership with senior policy-makers, researchers, and experienced practitioners can be applied in other settings to ensure training is responsive and customized to local needs, resources and priorities. Ongoing curriculum development should reflect international standards and be coupled with the development of appropriate performance support systems at the workplace for motivating staff to apply their newly acquired knowledge and skills effectively and creatively.
Citation: Wang Y, Li X, Yuan Y, Patel MS (2014) A Multi-Method Approach to Curriculum Development for In-Service Training in China’s Newly Established Health Emergency Response Offices. PLoS ONE 9(6): e100892. https://doi.org/10.1371/journal.pone.0100892
Editor: Linda A. Selvey, Curtin University, Australia
Received: October 3, 2013; Accepted: June 1, 2014; Published: June 27, 2014
Copyright: © 2014 Wang et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The project was funded through China’s National Health and Family Planning Commission’s “Special Fund for Health Scientific Research in the Public Interest [201002028]”, in collaboration with the China Office of the World Health Organisation. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The outbreak of severe acute respiratory syndrome (SARS) in 2003 was a watershed moment for China [1]. It triggered major health reforms [1]–[4] and led to an increase in public health funding of about 100% by 2007, accounting for a rise in spending from 0.75% to 0.89% of the gross domestic product [1]. Subsequent emergencies that echoed the imperative for reforms included the outbreaks of influenza H5N1 in birds and humans [1]–[4], melamine contamination of milk formula that affected over 294,000 Chinese children [5], and the earthquake in Sichuan that resulted in over 69 000 deaths, displaced about 15 million people and led to the mobilization of over 10 000 medical workers [6].
The reforms to strengthen China’s emergency preparedness and response capabilities included new laws and regulations [7]–[10], increased support for training and research, and the adoption of international best practice [2]–[4], [11]. To help implement the new laws, Health Emergency Response Offices (HERO) were established within China’s health administration units at the national, provincial and municipal levels, in Centers for Disease Control (CDC) and in tertiary hospitals [12]. The HEROs within any one province now collectively employ around 1000 staff members, and their role is to help develop and coordinate preparedness planning and emergency response within their jurisdictions, as well as implement the International Health Regulations (2005) (IHR) [13]. However, because staff members were recruited opportunistically from diverse professional backgrounds into the newly established HEROs, they had not been trained systematically in emergency preparedness and response previously. While they received in-service training, this was offered in an ad hoc manner, was not preceded by a needs assessment [14], and was aimed primarily at improving knowledge of the new laws that did not translate directly into strengthening performance of the HEROs [15], [16].
At the global level, perennial threats of the pandemic spread of infectious diseases like SARS and influenza, as well as the sequelae of earthquakes, tsunamis, bioterrorism and complex humanitarian emergencies, heightened awareness of the need to strengthen national, regional and global capacity in prevention, preparedness and response to public health emergencies. In 2004, the World Association for Disaster and Emergency Medicine (WADEM) proposed a framework for disaster health to facilitate the development of educational programs in the field [17]. Efforts were directed at identifying and defining criteria for assessing disaster health-related competencies, standards for guiding curricular development, and exploring the methods, duration and desired outcomes of training [17]–[19]. Australia for example, developed a national framework for disaster health education to provide guidance for educators to achieve a standardized and integrated approach across the country [20]. Australia adapted the WADEM recommendations to target seven educational levels, and outlined the curricular contents and outcomes for each level; the seven levels were defined as 1) informing the community, 2) raising awareness of health workers, 3) providing basic knowledge and skills for health workers, 4) advancing the knowledge of health workers who play lead roles in disaster management, 5) enhancing expert knowledge among a small group of health workers, 6) targeting specialist level amongst a small group of individuals, and 7) encouraging research and innovation to further develop the knowledge base of disaster health [20].
In recognition of the national need for an in-service program to target the “level four” health staff as defined by WADEM, i.e. “health workers who played lead roles in disaster health management” [20], China’s Ministry of Health (MoH) commissioned the Capital Medical University in 2010 to develop and implement a competency-based curriculum to help strengthen the performance of the new cadre of HERO staff. An immediate priority was to address the WHO requirements for countries to meet the core capacity requirements for implementing the IHR by June 2012 [21]. This paper outlines the consultative process used to conduct the training needs assessment and develop the curriculum for implementation across China, with support from WHO. The multi-method approach to curriculum development can be applied in other settings to ensure training is responsive and customized to local needs, resources and priorities.
Methods
Ethics Statement
We did not submit the study proposal for ethics approval because we conducted meetings and interviews with study informants on a voluntary basis for the sole purpose of identifying training needs at the workplace; we did not gather any personal information or attributes about individual informants beyond their age and past work experience.
Figure 1 outlines the approach for developing the in-service curriculum using ADDIE as the basic framework for instructional design [22]. Each of the five phases of this model – Assessment, Design, Development, Implementation, and Evaluation – were reviewed critically by a technical advisory panel (TAP). The panel offered suggestions for strengthening the curriculum development process, as well as the content, relevance and quality of the curriculum. The multi-method training needs assessment and subsequent steps of the ADDIE model are detailed below.
Assessment of training needs
We defined a competency as “a cluster or related knowledge, skills, and attitudes that reflects a major portion of one’s job (a role or responsibility), that correlates with performance on the job, that can be measured with well-accepted standards, and that can be improved with training and development” [23]. A competency domain incorporated an inter-related group of competencies for performing specified tasks.
In order to identify the list of competency domains against which training needs of HERO staff could be assessed, we collated data from three sets of references. The first was from the IHR core capacities necessary to detect, assess, report and respond to emergencies [21]. The second set of references was from inter-related Chinese laws and regulations: the Emergency Response Law which is the legal framework for managing emergencies [7], the National Health Emergency Training Outline which identifies the knowledge and skills to be targeted by training programs [11], and the Regulations on Preparedness for and Response to Emerging Public Health Hazards which outlines preparedness and response activities [8], [10]. The final set of references was selected from Chinese publications on general guidelines for training health workers [24] and from curricula for training technical staff based in disease surveillance units, laboratories and the environmental health sector [14]–[16], [24]–[27].
We assessed tasks, roles and responsibilities, and training needs of HERO staff through face-to- face interviews of eleven experienced key informants; they included health emergency experts from the government, the military and the academic sector, and senior staff of HEROs. We also explored their awareness of existing training activities and the associated relative strengths and weaknesses, their preferences on modes of curriculum delivery and the optimal duration of in-service training. We collected data on the same issues through self-administered questionnaires to a group of 115 HERO staff members from across China when they attended a training workshop in Beijing.
China’s Ministry of Health (MoH) appointed a technical advisory panel (TAP) to critically review and offer comments and suggestions on all aspects of the needs assessment and curriculum development process, as well as on the relevance and quality of the curriculum. The eight panel members were nationally acknowledged experts in public health emergencies from the MoH, CDCs, HEROs and the Academy of Military Medical Science.
Designing and developing the curriculum
We used the results of the needs assessment to design the draft curriculum that included the topics listed in Figure 1. Each of the identified competency domains was targeted through a self-contained module that could be run independently of the other modules. Each module encompassed theories, laws, concepts and tools and techniques essential for HERO staff to perform effectively in their work.
We developed questionnaires for HERO staff to invite their comments and suggestions on the curriculum design, and the expected level of need and potential demand for the training. The questionnaires were disseminated with the draft curriculum to 700 HERO staff members at the provincial and municipal levels and 1000 health staff responsible for responding to emergencies at the local level. We also invited comments on the curriculum from the Technical Advisory Panel and staff of the WHO Office in China. We used their feedback to revise the draft curriculum.
Implementing and evaluating the curriculum
The revised curriculum was endorsed by the Technical Advisory Panel and the Ministry of Health before it was implemented, monitored and evaluated between June and November 2010.
The key tasks of HERO staff in their respective jurisdictions were identified as coordination of the development and implementation of preparedness plans and early warning surveillance systems, mobilization of emergency response teams, and implementation of the IHR [13]. The list of competency domains associated with these tasks is shown in Table 1. Domains 1 to 8 were transcribed from both the IHR [13], [21] and the Chinese references [7], [10], [11]. Domains 9 to 18 were explicit only in the latter set of references; although implicit in the IHR capacities, study respondents suggested they be considered as discrete competency domains.
Training needs assessment
Of the 1700 health professionals to whom questionnaires were distributed, 1606 (94.5%) responded with comments and suggestions; selected characteristics of these respondents are summarized in Table 2. Of the respondents, just under two-thirds were aged 40 years and over, 45% had worked previously as clinical practitioners, 35% as preventive health (public health) practitioners, and 8% as health administrators. Overall, 58% had less than 5 years experience in emergency response work.
The major weaknesses the respondents, key informants and the Technical Advisory Panel identified in existing training programs were the didactic methods used for classroom teaching and the emphasis on hazard-specific approaches (such as for responding to SARS or pandemic influenza or earthquakes), and on acquiring knowledge of the new laws and regulations rather than on addressing the challenges to their implementation. Furthermore, while training programs in the past had targeted knowledge and skills required by technical staff such as epidemiologists, laboratory scientists and environmental staff, they had not incorporated skills in mobilizing, managing and coordinating teams across multiple sectors, disciplines and agencies. The respondents stressed the need for adopting interactive learning methods and problem-solving exercises in the classroom, as had been suggested in Chinese publications [26]–[29]. They supported replacing the ‘hazard-specific’ approach with the ‘all hazards’ approach to managing emergencies, development of a competency-based curriculum with modules that addressed specific sets of competency domains, and limiting the duration of in-service training to about 2 to 3 weeks so these interfered minimally with trainees’ regular work responsibilities.
Designing the draft curriculum
After discussions with the key informants and TAP, 11 of the 18 competency domains shown in Table 1 were to be developed into the individual modules shown in Table 3. They included six of the eight IHR-related competencies, while the remaining two, ‘national legislation’ and ‘national co-ordination’ were omitted because they were directed mainly at national-level decision-makers. The other five domains were specified in the Chinese references, namely on-site organization during response, decision-making processes for alerting the community, resource management and stockpiling, risk assessment and management, and evaluation of response to emergencies. The remaining seven domains shown in Table 1 were not developed into modules because they were considered to be a lower priority for health managers and could not be included in the first round of in-service training to be conducted over a limited 2–3 week timeframe.
The proportion of the 1606 respondents who expressed a high level of need for these modules are shown in Table 3.
Case studies and desktop simulation exercise were both ranked as the preferred training method by 94% of respondents, followed by lectures (85%) and case studies (85%). The median preferred time for each module was 2.5 days. To accommodate this request and complete the course within 3 weeks, we included only nine modules, and excluded modules for two domains considered not to be priorities for HEROs: human resources and laboratory.
Developing the curriculum
The curriculum was revised to incorporate feedback from the stakeholders. Two trainers prepared each module to be run over 2 to 2.5 days; one was typically a university academic with nationally acknowledged expertise in emergencies, and the other, a senior staff member of a HERO. Each module would be delivered through three sessions: a lecture on relevant theories, concepts and tools and techniques that were followed by a case study and a desktop simulation exercise; the latter two sessions were conducted with small groups of 6 to 8 participants as an opportunity to apply, experiment with, and consolidate the knowledge acquired from the lectures.
Before the course, trainees were to be provided with readings for each module, and invited to prepare appropriate material from their workplace for a case study relevant to at least one module; this could include a preparedness plan, report on a risk assessment, a risk communication plan, or other reports on an emergency event. The trainers then prepared case studies either from the participants’ material, or designed them de novo based on their own experiences. An example of the former was a case study based on a jurisdictional preparedness plan and where participants had to identify strengths, weaknesses and contentious issues, and to explore alternative options in the light of the newly acquired knowledge from the lectures. On returning home, they were expected to review and revise their jurisdictional plan using a similar learning approach with their local team members. An example of a case study designed by the facilitator was the assessment and management of the risk of an infectious disease outbreak at a conference attended by about 1000 international participants. Each small group would then be expected to present a 20 minute report for further discussions at a plenary, with a synthesis of key learning points.
An example of a desktop simulation exercise developed by the facilitator was an earthquake scenario where participants had to convene a national response team, to mobilize them within a safe operational base near the affected zone, and then arrange the logistics for transporting and storing essential medical supplies and equipment. Each small group would prepare and submit written responses to the evolving scenario to which they would receive feedback from the facilitators.
The delivery of the modules was to be monitored to document attendance of trainers and trainees, the content of the training materials and the mode of delivery of each module. The evaluation plan was derived from the four Kirkpatrick levels of evaluation [30]; the first two levels, ‘reaction’ to the learning content and environment and ‘learning’ (or acquisition of new knowledge) were both assessed immediately after each module. The next two levels, ‘behavior’ at the workplace and ‘results’ (or the benefits of change in behavior at the workplace) were assessed three months later.
The results of the needs assessment and curriculum were presented, debated and finalized at a workshop with key informants, TAP members and senior HERO staff.
Implementing and evaluating the curriculum
The first in-service course was delivered to a class of 37 HERO staff members from 31 provinces across China; each participant had at least one undergraduate degree and over two-year’s work experience in emergency response. The monitoring and evaluations activities were implemented as planned, and the results will be published separately.
Discussion
This study on developing an in-service curriculum for new professional cadre of HERO staff from across the country adds to the literature on China’s health reforms and enduring efforts to strengthen emergency response capabilities following on the SARS outbreak and other emergencies [1]–[4]. It describes the multi-method approach used to identify training needs systematically, and to adopt international best practice in partnership with senior decision-makers and content experts from the government, academia and the military. The study revealed the overwhelming need to replace didactic teaching, the classical method used across China, with active learning and training methods. The consultative process for developing the curriculum was designed to address the scale of the challenge for coordinating planning and training activities across jurisdictions that cover a population of over 1.3 billion people, and where most provinces have more than 60 million residents. The process required active engagement with experts in government, academia and the military, as well as inputs from 1606 immediate potential beneficiaries. The expert technical advisory panel played a critical role through each step in the ADDIE model, and endorsed the curriculum development process as well as the contents of the curriculum.
The IHR (2005) core capacities are aimed at minimizing the international impact of communicable disease emergencies and chemical, radiation and nuclear accidents [13], [21]. However, they are also essential for timely and effective detection of, and responses to, the emerging threat at its source. China’s reforms to strengthen preparedness and response capabilities [2]–[4], [7]–[11] therefore incorporated the IHR core capacities as well as the competency domains identified as priorities based on China’s recent experiences with emergencies [1]–[6].
The overwhelming rejection of didactic teaching methods by the study respondents and the need to replace these with active learning and teaching methods was encouraging news for the curriculum designers. This approach is consistent with the two Chinese symbols that depict “learning” (??: xué xí) - ‘acquisition of knowledge’ and ‘repeated practice’ as inseparable sides of the one coin. Training aimed at strengthening performance has to create opportunities for active learning and experimentation with problem solving exercises to apply and consolidate newly acquired knowledge and skills.
The curriculum marks the beginning of a new journey for strengthening the performance of the recently established HEROs in jurisdictions across China. This initial effort focused strongly on meeting the WHO requirements for developing the core capacities required to implement the IHR [21], and in particular, to strengthen the competencies of the “level four” health staff [20] who play lead roles in emergency planning and management. Our approach for identifying and developing competencies and standards are consistent with those published by WADEM in 2013 [18], [19], and in many ways resemble those used in Australia [20]. The initial list of 18 competency domains were derived from the eight core capacities required to implement the IHR [21] and the disaster management cycle of prevention, preparedness, response and recovery [17], both of which are embedded in Chinese laws and regulations [7]–[12]. The scope of the task and all aspects of the needs assessment and identification of the competencies were developed through iterative inputs from subject matter experts in the Ministry of Health, academia and the military. The mode of curriculum delivery was based on blended learning methods implicit in Bloom’s taxonomy as outlined in the Australian framework [20]. The duration of training for the level four workers In Australia is recommended for 40 hours [20]; by contrast, our course was conducted over 100 hours because the target group had received little training in the past and included workers with diverse sets of past experience. However, consistent with the WADEM guidelines [17]–[20], China needs to reconcile its curriculum with international standards and expand its efforts to develop and standardize training frameworks for the other six WADEM levels [20], as well as interconnect with the “cross-cutting” competencies of workers from the other disciplines that participate in emergency responses [19]. Adopting the international standards and practice will also enhance communications, inter-agency cooperation and inter-operability across China’s national borders.
The major limitation of our training is that it helps develop only the knowledge, skills and attitudes essential for working in the field of emergency preparedness and response. We recognize that training constitutes but one component of a package of activities needed to strengthen the performance of HERO staff. Training must be linked with an appropriate performance support system [31] that motivates staff to apply the newly acquired knowledge and skills, and in this way, to start contributing effectively and creatively to the workplace. The support system includes a congenial work environment; explicit guidelines, supervision as well as feedback on the quality of their performance; appropriate resources such as materials, equipment and computer software; and incentives such as financial and non-financial rewards for good performance as well as opportunities for career development [31].
The multi-method approach to curriculum development by engaging actively with senior policy-makers, researchers, and experienced practitioners can be applied in other country settings to ensure training is responsive and customized to local training needs, resources and priorities.
Acknowledgments
We thank Lizheng Guan, Rui Guo, Xingming Li, Jun Li, Nina Wu, Kai Meng for assistance in preparing the training materials; Shibing Yang, Wenwen Liu, Yiwen Yuan, Fei Teng, Ning Su, Wei Liu, Yiyi Kong for their assistance in implementing the curriculum; Yanping Mi, Bin Liu, Chongli Zhong, Bin Ma from the National Health and Family Planning Commission of the People’s Republic of China and CK Lee from the WHO Office in China for expert advice and support through all stages of curriculum development and implementation.
Author Contributions
Conceived and designed the experiments: YW XL MP. Analyzed the data: YW XL YY MP. Wrote the paper: YW XL MP. Conceived the approach to curriculum development, convened meetings with the technical advisory panel and key informants, coordinated the development of the data collection tools and analysis, and drafted the curriculum: YW XL YY MP. Developed the data collection tools, analysed the data and produced the first draft of the manuscript: XL YY. Assisted with data collection, analysis and interpretation of the data: YY. Provided the evidence to support the study methodologies and led revisions of subsequent drafts of the manuscript: MP. Contributed intellectually to the paper and have seen and approved the final version: YW XL YY MP.
References
- 1. Chan LH, Chen L, Xu J (2010) China’s engagement with global health diplomacy: was SARS a watershed? PLoS Med 7: e1000266.
- 2. Hipgrave D (2011) Communicable disease control in China: From Mao to now. J Glob Health 1: 224–238.
- 3. Wang L, Liu J, Chin DP (2007) Progress in tuberculosis control and the evolving public-health system in China. The Lancet 369: 691–696.
- 4. Wang L, Wang Y, Jin S, Wu Z, Chin DP, et al. (2008) Emergence and control of infectious diseases in China. Lancet 372: 1598–1605.
- 5. Ingelfinger JR (2008) Melamine and the global implications of food contamination. N Engl J Med 359: 2745–2748.
- 6. Zhang L, Liu X, Li Y, Liu Y, Liu Z, et al. (2012) Emergency medical rescue efforts after a major earthquake: lessons from the 2008 Wenchuan earthquake. The Lancet 379: 853–861.
- 7.
Order of the President of the People’s Republic of China (No. 69) (August 2007). Emergency Response Law of the People’s Republic of China. In Chinese.
- 8.
Order of the State Council of the People’s Republic of China (February 2006). National emergency plan for responding to public health emergencies. In Chinese.
- 9.
Order of the State Council of the People’s Republic of China (January 2006). National master plan for responding to public emergencies. In Chinese.
- 10.
Order of the State Council of the People’s Republic of China (No. 376) (May 2003). Regulations on Preparedness for and Response to Emergent Public Health Hazard. In Chinese.
- 11.
Ministry of Health of the People’s Republic of China (October 2006). The 2006–2010 National Health Emergency Training Outline. In Chinese.
- 12.
Ministry of Health (2004). The major points of work to health emergency office in 2004. In Chinese.
- 13.
WHO (2005) Fifty-eighth World Health Assembly resolution WHA58.3: Revision of the International Health Regulations. Geneva: WHO.
- 14.
Hu G-q, Rao K-q, Sun Z-q, Yu R-h (2008) Assessment of capacity for logistics, public education and training in public health emergency management in China Journal of Central South University (Medical Sciences) 10: 947–951. In Chinese.
- 15.
Ning N, Wu Q-h, Li B, et al.. (2008) Investigation and Analysis on Current Status of Emergency Training to Staffs of Centers for Disease Control and Prevention in Heilongjiang Province Chinese Health Economics 307: 66–69. In Chinese.
- 16.
Xue L, Wang Z-Q, Peng Z-Q, et al.. (2011) The Current Situation and Development of Emergency Management Training System in China China Emergency Management 8: 12–17. In Chinese.
- 17. Archer F, Seynaeve G (2007) International guidelines and standards for education and training to reduce the consequences of events that may threaten the health status of a community. Prehosp Disaster Med 2007 22: 120–130.
- 18.
Daily E, Williams J (2013) White Paper on identifying and assessing competencies in disaster health World Association for Disaster and Emergency Medicine. Available: http://www.wadem.org/documents/competencies_in_disaster_health.pdf. Accessed 11 May 2014.
- 19.
Shultz J, A L (2013) White Paper on setting standards for selecting a curriculum in disaster health World Association for Disaster and Emergency Medicine. Available: http://www.wadem.org/documents/curriculum_in_disaster_health.pdf. Accessed 11 May 2014.
- 20. FitzGerald G, Aitken P, Arbon P, Archer F, Cooper D, et al. (2010) A National Framework for Disaster Health Education in Australia. Prehosp Disaster Med 25: 4–11.
- 21.
WHO (2010) IHR Monitoring Framework: Checklist and indicators for monitoring progress in the development of IHR core capacities in States Parties. WHO, Geneva 2010.
- 22.
Morrison GR (2010) Designing Effective Instruction. 6th Edition: John Wiley & Sons.
- 23. Parry S (1996) The Quest for Competence. Training Magazine 33: 48–56.
- 24.
Liu Y-G, Huang J (2006) Guidelines for Training Health Workers. Beijing: China Financial & Economic Publishing House. In Chinese.
- 25.
Un L-m, Liu Y-f, Luo H-m (2006,) Investigation on demand for public health emergency system of health workers in Guangzhou. Journal of Public Health and Preventive Medicine, 17: 10–13. In Chinese.
- 26. Wang C, Wei S, Xiang H, Xu Y, Han S, et al. (2008) Evaluating the effectiveness of an emergency preparedness training programme for public health staff in China. Public Health 122: 471–477.
- 27. Wang C, Xiang H, Xu Y, Hu D, Zhang W, et al. (2010) Improving emergency preparedness capability of rural public health personnel in China. Public Health 124: 339–344.
- 28.
Gao L, Wang Q (2011) The survey on China adult teaching mode study in recent twenty years. Journal of Adult Education of Hebei University 03: 21–23. In Chinese.
- 29.
Wang Z (2005) Adult Teaching Methods and Patterns. Journal of Hulunbeier College 6. In Chinese.
- 30.
Kirkpatrick DL, J.D K (2006) Evaluating training programs: The four levels San Francisso: Berrett-Koehler Publichers Inc.
- 31. Chevalier R (2003) Updating the behavior engineering model. Performance Improvement 42: 8–14.