Quantum Opportunities Program Evaluation Summary

PILOT PROGRAM

In 1989, researchers from Brandeis University, led by Eisenhower Trustee Andrew Hahn, began the evaluation of the first group of Quantum Opportunities Programs (not Eisenhower Foundation sponsored). Their findings clearly demonstrated the benefits of the Quantum model.

Study Methodology

Program designers randomly assigned 50 disadvantaged students in each of the five sites (Philadelphia, PA; Saginaw, MI; Oklahoma City, OK; San Antonio, TX; and Milwaukee, WI.) to either a program or a control group. Researchers compared the progress of the two groups with periodic questionnaires and basic skills tests. Due to the failure to implement at one program site (Milwaukee), the analysis was completed on four of the original five sites.

Findings

The first group of QOP sites were fully funded and the model rigorously followed. Brandeis researchers evaluated the four QOP sites. Relative to a control group, QOP students:

graduated from high school more often (63 vs.42 percent)
dropped out of school less often (23 vs. 50 percent)
went on to postsecondary education more often 42 vs. 16 percent)
attended a 4-year college more often (18 vs. 5 percent)
attended a 2-year institution more often (19 vs. 9 percent)
became teen parents less often (24 vs. 38 percent)
more often:

took part in a community project in the six months following QOP (21 vs. 12 percent);
were volunteer tutors, counselors or mentors, (28 vs. 8 percent) and
gave time to non-profit, charitable, school or community groups (41 vs. 11 percent, only statistically significant at the Philadelphia site)

Contributing Factors

The Brandeis report concluded that the key contributing factors in the success of the program were:

Caring Adults
“If young people are connected with caring adults for sustained periods of time, year-round, positive results do emerge.” Program administrators and staff, as well as teachers and mentors, took an active interest in the welfare of the QOP students, encouraging them, visiting them, following up and doing everything they could to keep them in the program. “Once in QOP, always in QOP” was the unofficial motto, and most program counselors took it to heart.

Sense of Community
The project sites were small, with only 25 students in each. Students were able to bond with each other and with adults in the program, particularly at the Philadelphia site.
“Simply put, when a quantum opportunity was offered, young people from public assistance backgrounds--African American males, females, whites, Asians, others -- took it! They joined the programs and many stayed with the programs or the staff associated with the initiatives, for long periods.”

Multiple Services Encompassing All Aspects of Youths’ Lives
The QOP program was designed to address the many challenges and obstacles that disadvantaged youth face. QOP focused on developing basic skills (academic and functional) for future success, strengthening life and social skills to make better choices and operate more effectively with families and peers, broadening horizons through cultural trips and other experiences, and taking pride in the community through active service.

Quality Staff
Results from the most effective project site--Philadelphia--show what can be accomplished with a dedicated, quality staff. “The differences, for example, between San Antonio and Philadelphia cannot be attributed to the neighborhood setting, the characteristics of participants, or to the program model. What distinguishes these sites is the degree of buy-in from the host organizations and the commitment of staff at all levels.”

Financial Incentives As Part of a Comprehensive Program
While financial incentives were important to some students, and helped with family expenses, it appeared that they were not the decisive factor in QOP participation. When they are part of a comprehensive, well-developed program, financial incentives can be effective in maintaining student interest in and attendance at program events. However, they do not appear to operate effectively in the absence of a strong program featuring much personal contact with staff.

Financial Resources
The Ford Foundation funded the QOP program upfront, making it possible to plan for and deliver a host of services over an extended period of time. Both staff and students knew the resources were there to carry through on their commitments.

DEMONSTRATION PROGRAM

Following this initial success, Mathematica Policy Research, Inc. evaluated the demonstration programs, sponsored by the U.S. Department of Labor and the Ford Foundation. The results from this demonstration project were much less impressive, due in large part to the failure of the sites to accurately implement the model, that had been shown to work in pilot phase.

YOUTH TODAY REVIEW

In an article entitled The Best Youth Program You Can Not Afford, Youth Today discussed both the problems and the benefits of the Quantum Opportunities model.

EISENHOWER FORUM

In November 2003, the Milton S. Eisenhower Foundation (MSEF) convened a group of evaluators, funders, and directors of QOP demonstration and replication sites to a two-day forum. The meeting focused on practical questions about how best to implement QOP and how to skillfully replicate the program on a larger scale. Their insights are telling, not only for providers of the QOP model, but to anyone involved in youth development. To read a summary report of the 2003 QOP Forum click here.

EISENHOWER'S FIRST COHORT

The Foundation implemented Quantum in four locations in Fall 2002. As of June 2006, all four cohorts have completed the four year program. Using a comparison group methodology, much like the original Brandeis study, data have been collected over the four year period. While the final analysis of the findings is not yet complete, preliminary findings demonstrate that these four sites were at least as successful as the original pilot sites, with higher graduation rates, lower drop-out rates, lower teen parenting and lower involvement with the law.