The flexibility federal block grants provide recipients poses challenges for evaluation. These challenges include aggregating data on wide-ranging activities grant recipients implement and the outcomes they achieve. In 2014, we began designing an evaluation to address the challenges of assessing outcomes and to improve outcome accountability for the Preventive Health and Health Services Block Grant. Through the use of evaluability assessment methodology, review of existing data and the literature, and key informant interviews, we developed a measurement framework to assess outcomes resulting from recipients’ ability to use grant funds to meet their locally prioritized needs. We argue our evaluation approach demonstrates that block grants, and other similarly flexible programs, can be evaluated through appropriately designed measures. Our efforts challenge the idea that flexibility presents an insurmountable barrier to evaluation and outcome accountability for federal block grants.

Objectives. To describe the career trajectories of 1 cohort of US Public Health Associate Program (PHAP) alumni over 3 years since completing PHAP. Methods. We distributed a Web-based survey at 3 time points between 2014 and 2017 (response rate = 76%). We calculated descriptive statistics in SPSS. Results. At all time points, most alumni were employed. Of those, the percentage employed in public health was 100% at program completion, 86% at year 1, and 68% at year 3. Conclusions. Most alumni were employed in public health jobs at each time point. At the 3-year mark, approximately a third of the alumni had left public health employment, which is in line with documented rates of turnover within the broader public health workforce. Public Health Implications. Service learning programs like PHAP are effective at recruiting early career professionals into public health. The extent to which PHAP is effective at retaining workers in public health after the program appears most promising immediately following the program or in the short term after the program concludes. The extent to which workers are retained in the longer term requires further study. (Am J Public Health. Published online ahead of print July 18, 2019: e1-e3. doi:10.2105/AJPH.2019.305156).

There is currently a gap in the literature regarding the creation of psychometrically sound measurement tools assessing service-learning programs in health-related fields. Without comprehension of a survey's psychometric properties, evaluators cannot ensure that survey instruments are reliable or valid. This study describes the psychometric evaluation of the Public Health Associate Program (PHAP) Service-Learning Scale (PSLS). PSLS assesses participant experience in PHAP, a Centers for Disease Control and Prevention program. This paper explains survey development, scale validity and reliability, and the internal factor structure of the PSLS. The final scale consisted of 22 items with a high internal consistency (Cronbach's α=.90). Exploratory Factor Analysis (EFA) was used to determine the scale's factor structure; five factors comprising of all 22 items were retained. The factors, or subscales, were Learning Outcomes, Mentoring, Experiential Assignment, Self-Efficacy in Program Competency Domains, and Program Satisfaction. All were also found to have adequate internal consistency (Cronbach's α >.70). Service-learning is vital in developing the next generation of the workforce. These study findings suggest the PSLS fills a critical gap in the literature by providing a valid and reliable instrument to evaluate experiences and satisfaction in service-learning programs and other fellowships.

The "learn by doing" approach to training is common in the public health field and is a core component of service-learning programs. Trainee satisfaction, learning, and application of learning have been studied. What is less understood is the perspective of the agencies that host trainees. This study aimed to identify whether and how the Centers for Disease Control and Prevention's Public Health Associate Program (PHAP) adds value to the agencies that host trainees during 2-year field assignments. An exploratory, qualitative study design consisting of 9 semistructured telephone interviews with PHAP host agency supervisors was used. Results suggested that PHAP increased host agencies' capacity by assigning capable trainees to host agencies. Trainees made quality contributions that led to agency- and/or community-wide improvements and positively affected the agencies' culture. Further evaluation of the host perspective is necessary; as coupled with the trainee's perspective, it will provide a more holistic understanding of program value.

OBJECTIVE: The Centers for Disease Control and Prevention (CDC) created the Public Health Associate Program (PHAP) to establish a continuous source of public health professionals who can deliver frontline services at the federal, state, tribal, local, and territorial levels. The article describes preliminary evaluation findings for PHAP. DESIGN: The evaluation's primary purposes are to assess the quality and effectiveness of PHAP, determine its value and impact, and provide information to continuously improve the program. Because the evaluation is both formative and summative and focuses on aggregate outputs and outcomes of PHAP, the methodology is complex and builds over time as different cohorts cycle into and out of the program. Results presented are outcomes of various Web-based surveys and reporting systems. PARTICIPANTS: Four PHAP cohorts, consisting of 579 individuals, participated in 1 or more of the evaluation activities described in this article. RESULTS: The majority of participants report satisfaction with their PHAP experiences, and 74% of recent graduates indicate they are continuing their careers or education in public health immediately after program completion. Seventy-eight percent of recent PHAP graduates who accept a job in public health are employed by the federal government. One year post-PHAP, 74% of alumni report that PHAP has been influential in their careers. CONCLUSION: CDC's investment in PHAP has increased the capacity and capabilities of the public health workforce. Results presented are early indicators of program quality, effectiveness, and impact. Today's public health workers are asked to do more with less, in the face of a dynamic array of complex public health challenges. PHAP offers public health agencies assistance in tackling these losses and challenges.

CONTEXT: Increasing the adoption and implementation of evidence-based policies and practices is a key strategy for improving public health. Although there is widespread agreement about the importance of implementing evidence-based public health policies and practices, there are gaps between what has been shown to be effective and what is implemented at the state level. OBJECTIVE: The Centers for Disease Control and Prevention (CDC) developed the Prevention Status Reports (PSRs), a performance measurement system, to highlight evidence-based public health policies and practices and catalyze state performance and quality improvement efforts across the nation. DESIGN: CDC selected a set of 10 topics representing some of the most important public health challenges in the nation. Stakeholders, including state health departments and other partners, helped conceptualize the PSRs and informed the development of the PSR framework, which provides an organizational structure for the system. CDC subject matter experts developed criteria for selecting policies and practices, indicators for each policy and practice, and a criteria-based rating system for each indicator. PARTICIPANTS AND SETTING: The PSRs were developed for all 50 states and the District of Columbia. MAIN OUTCOME: The PSRs were developed and serve as a performance measurement system for monitoring the adoption, reach, and implementation fidelity of evidence-based public health policies and practices nationwide. RESULTS: The PSRs include 33 policy and practice indicators across the 10 health topics. They use a simple 3-level rating system-green, yellow, and red-to report the extent to which each state (and the District of Columbia) has implemented the policy or practice in accordance with supporting evidence or expert recommendations. Results from aggregate analyses show positive change or improvement. CONCLUSION: The PSRs are a unique part of CDC's work to improve the performance and accountability of the public health system, serving as both a monitoring tool and a call to action to improve health outcomes. The PSRs can be used to track the reach of and fidelity to evidence-based policies and practices nationally over time, as well as inform state efforts to improve their use of evidence-based policies and practice.

OBJECTIVE: This study reports the use of exploratory factor analysis to describe essential skills and knowledge for an important segment of the domestic public health workforce-Centers for Disease Control and Prevention (CDC) project officers-using an evidence-based approach to competency development and validation. DESIGN:: A multicomponent survey was conducted. Exploratory factor analysis was used to examine the underlying domains and relationships between competency domains and key behaviors. The Cronbach alpha coefficient determined the reliability of the overall scale and identified factors. SETTING AND PARTICIPANTS: All domestic (US state, tribe, local, and territorial) grantees who received funding from the CDC during fiscal year 2011 to implement nonresearch prevention or intervention programs were invited to participate in a Web-based questionnaire. MAIN OUTCOME MEASURE(S): A total of 34 key behaviors representing knowledge, skills, and abilities, grouped in 7 domains-communication, grant administration and management, public health applied science and knowledge, program planning and development, program management, program monitoring and improvement, and organizational consultation-were examined. RESULTS: There were 795 responses (58% response rate). A total of 6 factors were identified with loadings of 0.40 or more for all 34 behavioral items. The Cronbach alpha coefficient was 0.95 overall and ranged between 0.73 and 0.91 for the factors. CONCLUSIONS: This study provides empirical evidence for the construct validity of 6 competencies and 34 key behaviors important for CDC project officers and serves as an important first step to evidence-driven workforce development efforts in public health.

INTRODUCTION: Continuous quality improvement is a central tenet of the Public Health Accreditation Board's (PHAB) national voluntary public health accreditation program. Similarly, the Centers for Disease Control and Prevention launched the National Public Health Improvement Initiative (NPHII) in 2010 with the goal of advancing accreditation readiness, performance management, and quality improvement (QI). OBJECTIVE: Evaluate the extent to which NPHII awardees have achieved program goals. DESIGN: NPHII awardees responded to an annual assessment and program monitoring data requests. Analysis included simple descriptive statistics. SETTING: Seventy-four state, tribal, local, and territorial public health agencies receiving NPHII funds. PARTICIPANTS: NPHII performance improvement managers or principal investigators. MAIN OUTCOME MEASURE(S): Development of accreditation prerequisites, completion of an organizational self-assessment against the PHAB Standards and Measures, Version 1.0, establishment of a performance management system, and implementation of QI initiatives to increase efficiency and effectiveness. RESULTS: Of the 73 responding NPHII awardees, 42.5% had a current health assessment, 26% had a current health improvement plan, and 48% had a current strategic plan in place at the end of the second program year. Approximately 26% of awardees had completed an organizational PHAB self-assessment, 72% had established at least 1 of the 4 components of a performance management system, and 90% had conducted QI activities focused on increasing efficiencies and/or effectiveness. CONCLUSIONS: NPHII appears to be supporting awardees' initial achievement of program outcomes. As NPHII enters its third year, there will be additional opportunities to advance the work of NPHII, compile and disseminate results, and inform a vision of high-quality public health necessary to improve the health of the population.

The Centers for Public Health Preparedness (CPHP) program was a five-year cooperative agreement funded by the Centers for Disease Control and Prevention (CDC). The program was initiated in 2004 to strengthen terrorism and emergency preparedness by linking academic expertise to state and local health agency needs. The purposes of the evaluation study were to identify the results achieved by the Centers and inform program planning for future programs. The evaluation was summative and retrospective in its design and focused on the aggregate outcomes of the CPHP program. The evaluation results indicated progress was achieved on program goals related to development of new training products, training members of the public health workforce, and expansion of partnerships between accredited schools of public health and state and local public health departments. Evaluation results, as well as methodological insights gleaned during the planning and conduct of the CPHP evaluation, were used to inform the design of the next iteration of the CPHP Program, the Preparedness and Emergency Response Learning Centers (PERLC).