The Program Being Evaluated
Achieving Competence Today (ACT) was a program of Partnerships for Quality Education, a national initiative of the Robert Wood Johnson Foundation (RWJF). ACT tested the use of a Web-based curriculum by medical residents and graduate nursing students at more than 20 schools as a means for them to identify problems in patient care quality and collaboratively use systems-based approaches to improving care. RWJF funded ACT from May 2003 through September 2008.
About the Evaluation
In 2005, RWJF funded evaluators Michael Yedidia, Ph.D., and Colleen Gillespie, Ph.D., of Rutgers University to carry out an evaluation of ACT through a post-training survey of ACT II learners, and pre- and post-training study of ACT III learners and leaders.
The evaluators sought answers in four areas:
- How did learners assess the value of participating in the quality improvement process in the context of their training?
- What was the nature of the quality improvement projects conducted as part of ACT training? How were the projects valued by learners and leaders? How did they assess the contribution of ACT to quality improvement at their institutions?
- To what extent did learners collaborate with peers or staff from different disciplines in developing improvement projects? How did learners and leaders assess the quality of the collaborative experience? What were the factors that either facilitated or impeded effective interdisciplinary collaboration?
- What factors appeared to differentiate those health professions education programs prepared to pay $10,000 to continue participation in ACT from those that didn't?
Summary of Methods
To carry out the evaluation, Yedidia and Gillespie conducted:
- A post-training survey of the ACT II cohort of learners to elicit their assessments of the ACT training and their experience with and evaluation of quality improvement projects.
- A survey of leaders of the ACT initiatives to assess their valuation of the learning experience; their appraisal of the contribution of the projects to quality improvement at their institutions; and their estimation of the centrality of interprofessional collaboration to the ACT initiatives.
- A pre-training and post-training survey of ACT III learners and a survey of ACT III leaders (faculty preceptors, medical education or quality improvement directors) to yield data for analyzing determinants of their decisions as to whether or not to continue participation with no funding.
Knowledge and Impact
In a report on ACT submitted to RWJF in January 2007, the evaluation team reported the following:
- Learners (medical residents or graduate nursing students) commended the ACT training overall. Some 80 to 89 percent of ACT learners "strongly agreed" or "somewhat agreed" that they would recommend the initiative to others.
- Among key elements of ACT training, learners most highly rated the development of a quality improvement project. Content areas most frequently chosen by learners for their projects were:
- Effectiveness of care—providing appropriate care to those who could benefit and avoiding unnecessary care.
- Timeliness—reducing patient wait times and potentially harmful delays.
- Patient safety.
- ACT leaders (medical education or quality improvement directors, ACT preceptors and teachers) concurred with learners in their assessment of the value of learners' quality improvement projects to their institutions. More than 88 percent of leaders and learners felt they were of moderate or great value.
- Interdisciplinary collaboration was viewed, generally, as very important to the work of quality improvement, with 65 percent of leaders and 69 percent of learners holding this view. Learners, regardless of discipline, credited their ACT training with improving their outlook on two key dimensions of interdisciplinary work:
- Developing trust among collaborators.
- Becoming more aware of the limits of a single-disciplinary approach to care.
- When asked what most facilitated collaboration, learners' most prominent response was "the necessity of getting the job done."
Learn how to improve care transitions and prevent avoidable hospital readmissions, and pick up nursing and medical education con-ed credits.
The RWJF Roadmaps to Health Prize honors outstanding community partnerships which are helping people live healthier lives. The six winners w...
Mildred Dalton Manning, the last surviving member of a group of U.S. Army and Navy nurses taken prisoner in the Philippines at the start of ...
A study finds that 96 percent of nurse practitioners and 76 percent of physicians agreed with IOM report recommendation that “nurse practiti...
The strange pull of this series is its humanity, not its horrors.
The Robert Wood Johnson Foundation is working to increase awareness and understanding of the impact of ACEs and the need to develop effectiv...
The What's Next Health series features leading thinkers and visionaries. Jake Porway, a data scientist and machine-learning enthusiast, talk...
These reports provide an important starting point for the next round of serious health care reforms.
A national conversation highlighting efforts to improve care transitions, reduce avoidable hospital readmissions, and lift overall quality o...
Playworks improving the health and well-being of children through safe, meaningful play
The reconvened Commission to Build a Healthier America will provide new guidance in two key areas: early childhood and healthy communities.
Headlines include: Psychiatrist "Bible" Gets a Numeric Overhaul Most Adults Enforce Smoke-Free Rules in Homes, Cars Living Near Fast-Food Ou...