The purpose of this paper is to summarize data from available evaluations of community school initiatives. Part One provides a vision of community schools, discusses the nature and limitations of existing research, and offers a summary of findings from 49 community school initiatives. Three examples of evaluations are included in Part One in order to illustrate the scope of efforts underway. Part Two describes and offers research data on 49 different initiatives. It is our hope that the dissemination of this preliminary report will encourage others to come forward with documentation of their efforts and encourage a stronger emphasis on research and the collection of results data.

What a Community School Looks Like

The first question usually asked about community schools is "what are they?" and the second is "do they work?" 'Community school' is an inclusive term, encompassing a growing number of school-community initiatives that feature both common themes and differing approaches. The names of the initiatives suggest some of the varying attributes: Caring Communities, Beacons, Bridges to Success, University-Assisted Schools, Healthy Start, Communities in Schools, School of the 21st Century and many others. Some are broad in scope, promoting widespread replication or adaptation, while others are single entities. Programs are being initiated at the national level (e.g., Children's Aid Society, School of the 21st Century), state level (e.g., New Jersey School-Based Youth Program), local level (e.g., Polk Bros. Foundation Full Services Schools Initiative in Chicago) and in individual schools (Molly Stark Community School, Bennington, Vermont). Community schools also vary in their goals: Some specifically aim to improve academic achievement while others focus primarily on health and behavioral outcomes or enhanced family functioning.

Although the Coalition for Community Schools recognizes that each community school is unique, it has developed a general description of a well-developed community school:

A community school, operating in a public school building, is open to students, families and the community before, during, and after school, seven days a week, all year long. It is jointly operated through a partnership between the school system and one or more community agencies. Families, youth, principals, teachers and neighborhood residents help design and implement activities that promote high educational achievement and positive youth development.

The school is oriented toward the community, encouraging student learning through community service and service learning. A before and after-school learning component encourages students to build on their classroom experiences, expand their horizons, contribute to their communities, and have fun. A family support center helps families with child rearing, employment, housing, immigration, and other services. Medical, dental, and mental health services are readily available. College faculty and students, business people, youth workers, neighbors, and family members come to support and bolster what schools are working hard to accomplish - ensuring young people's academic, interpersonal, and career success.

Ideally, a full-time community school coordinator works in partnership with the principal. This person is responsible for the delivery of an array of supports provided by local agency partners and participates on the management team for the school. Over time, most community schools consciously integrate activities in several areas to achieve the desired results: quality education; positive youth development; family support; family and community engagement in decision-making; and community development.

Research on effectiveness

Policy makers and practitioners want to know what kinds of positive changes community schools can make. Before addressing this question, we first ask, "Given our vision, what indicators of success should we be looking for?"

We believe that community schools should be seen as vehicles for education reform; therefore, improved learning and achievement must be a long-term measure of the effectiveness of this growing movement. In addition to test scores, learning and achievement related indicators include rates of attendance, promotion, graduation, suspension and expulsion for example.

It is important to note that community schools are designed to affect not only educational outcomes but other outcomes as well. Such outcomes include improved social behavior and healthy youth development; better family functioning and parental involvement; enhanced school and community climate; and access to support services. These outcomes have value in and of themselves, in addition to affecting educational outcomes.

The data summarized here indicate that community school initiatives are moving in the right direction across many different indicators of success, giving us cause for optimism. The story is not complete however. First, it should be noted that the programs we identified cover a broad continuum, from highly developed community schools that have been in existence for a decade or longer to schools that are just beginning to open their doors and offer expanded opportunities such as after-school activities. Second, the quality of the studies varied enormously, from evaluations that relied on very small non-representative samples to those that were based on carefully designed management information systems and control groups. At best, evaluation is difficult, expensive, and long-term. Only a few programs can produce what would pass as "scientific" results. Many others can offer "preliminary findings", early returns on long-term projects.

Limitations of the research must be understood. Even as we recognize some of the constraints we also must acknowledge the complexity of evaluation in a community schools context. Very few of the research studies summarized here used random assignment to define comparison groups. In the places that used comparison groups, these were either matched schools or non-participants. Randomized studies are extremely difficult, expensive, and time-consuming, and it is particularly challenging to identify "control groups." Many of the studies relied only on pre-post tests of participants, and then lost a substantial number of the students during the year or two that elapsed, in part due to family mobility. A few programs measured progress by achieving goals that may have been quite marginal. For example, one program had a goal that 40% of the students should improve their grade point average in a year. Reports of evaluations sometimes were only limited to positive results. Others presented tables of findings when it was clear that no effects were observed. In sum, these reports are preliminary and inconclusive, but the volume is impressive.

All around the country, researchers are struggling to do a better job of documenting the effects of these efforts. The constraints are many when one is trying to track events in an innovative multi-faceted program housed in a setting like a school where it is often difficult to conduct surveys. Researchers experience barriers to obtaining permission from families to survey their children and resistance from parents to filling out surveys. The students in disadvantaged schools have extremely high rates of mobility; half of the study population has left at the end of a year. School personnel often do not want to be bothered with filling out surveys and may not be invested in the findings. Staff may not know how to use the research findings for program improvement. In addition, funders often press for results too early in the process.

Perhaps we should listen more to Lisbeth Schorr and Daniel Yankelovich, who have argued eloquently for moving ahead with social programs and not getting bogged down in methodological warfare. "Evaluating complex social programs is not like testing a new drug. The interventions needed to rescue inner-city schools, strengthen families, and rebuild neighborhoods are not stable chemicals manufactured and administered in standardized doses. Promising social programs are sprawling efforts with multiple components requiring constant mid-course corrections, the active involvement of committed human beings and flexible adaptation to local circumstances." As you browse through the examples presented below, you will see just how "sprawling" this emerging community school field is and how difficult it is to capture all that is happening.

What is most important is that there is a growing body of evidence that community schools are beginning to demonstrate positive effects on students, families, and communities. The preliminary data summarized here suggest that many of these models have the capacity to produce multiple impacts that include, and go beyond the expectations of traditional education reform.

Sources of Information

This is a first cut at compiling documentation on the impact of community schools. Many evaluations are currently underway, and within the next several years we should have access to much more information. An examination of citations in this paper will confirm that the "literature" about community schools is mostly "hidden". Compiling the information involves extracting annual reports and unpublished documents from researchers and program administrators throughout the country. Web sites (identified where known) are a great benefit because one can download reports that are difficult to obtain in hard copy and have not been (and probably never will be) published.

We were able to obtain information on 49 school-community programs that have produced recent evaluation reports or data on results. No two initiatives are alike nor are the research protocols used. Of the 49, six are after-school initiatives such as LA's Best. Some research reports, for example on the large-scale California Healthy Start Initiative, aggregate findings for hundreds of schools. Other programs, such as Communities in Schools, have submitted reports at state, community, and local site levels (counted as one report). Other evaluation efforts focus exclusively on one school such as the study of the Broad Acres Elementary Linkages to Learning School in Montgomery County, Maryland. Thus, the count of reports is rough and is intended to serve as an estimate of the large quantity of information that is becoming available.

All of the reports present findings on one or more outcomes, such as improved academic achievement, change in student behavior, or increased parental involvement. In 46 of the reports some positive changes were noted. In three of the reports, no positive changes were reported - that is, students in the schools with the programs did no better than those in comparison schools, or alternatively, students in the program did not improve their performances or behaviors over time.

Key Results

Highlights of the compilation on outcomes are presented below. The outcomes are organized into four major categories which reflect the comprehensiveness of community schools: learning and achievement, improved social behavior and healthy youth development; better family functioning and parental involvement; and, enhanced community life. Specific indicators are set forth for each outcome.

Learning and Achievement

Achievement: Thirty-six of the 49 programs reported academic gains. These gains generally included improvements in reading and math test scores, looked at over a two- or three-year period. Many of the programs reporting academic gains were in elementary schools. In at least eight of the cases, the outcomes were not schoolwide. Rather, they were limited to students who received special services, such as case management, intensive mental health services, or extended day sessions.

However, there were some instances of school-wide improvements on academic measures:

• Charles Drew Elementary School, a participating school in University of Pennsylvania's West Philadelphia Improvement Corp program, showed more improvement on the state's standardized reading and math tests than any other school in the state in 1999, an increase of 420 points.

• At PS 5, a Children's Aid Society Community School in New York City, the percentage of children reading at grade level rose from 28% when they were in grade 4 to 42% by the time they reached grade 6.

Attendance: Nineteen programs reported improvements in school attendance. Several reported lower dropout rates, one specifically among pregnant and parenting teens. Several mentioned higher teacher attendance rates, suggesting higher levels of satisfaction. Examples of positive results in this area:

• A national evaluation of Communities in Schools found that about 70% of students with high absenteeism prior to participation in CIS improved their attendance.

• At Lane Middle School, a Schools Uniting Neighborhoods sites in Portland OR, attendance increased from 85% to 91% over two years.

Suspensions: Eleven programs reported a reduction in suspensions. This may reflect changes in suspension policies, rather than changes in behaviors leading to suspensions. As schools transform into more child-centered institutions, they are likely to change practices regarding suspensions and expulsions as part of the change in school climate.

• At the Woodrow Wilson Middle School in Des Moines, Iowa the rate of suspensions in 1995 was one-sixth the rate five years earlier.

• At Lane Middle School in Portland, OR, suspensions declined from 50 to 15 over two years.

Social Behavior and Healthy Youth Development

High-risk behaviors: Eleven programs reported reductions in rates of substance abuse, teen pregnancy, disruptive behavior in the classroom, or improvement in behavior in general. For example:

• An evaluation based on 138 grantees in California's Healthy Start initiative found that students receiving Healthy Start services decreased their drug use.

• The Blenheim School (Missouri Caring Communities site) reported a 40% decrease in disruptive behavioral incidents following the initiative of a system for referrals for clinical therapy.

Parent involvement. At least 12 of the programs reported increases in parent involvement. For example, at the Bryant School, a Caring Communities site in Missouri with an intensive family intervention program, volunteer hours increased from 43 in 1996 to 2,008 in 1998.

Family functioning. In many programs with a strong family focus, improved family functioning was reported.

• Parents who received School of the 21st Century services were able to improve their child development practices, were less stressed, spent less money on child care, and missed fewer days work.

• An evaluation of 138 grantees of California's Healthy Start initiative found that families reported improvement in filling basic needs such as housing, food and clothing, transportation, finances and employment.

Community Life

Access to services: Better access to health care, lower hospitalization rates, higher immunization rates, or access to dental care were reported at least once. After-school programs cited access to child care as a significant outcome.

• As a result of putting full time school nurses in school sites in the Success Program in Des Moines Public Schools, 97% of the children were immunized. Dental screenings produced a 34% increase in improved oral hygiene and a 36% decrease in referrals for cavities.

• At Broad Acres Elementary School in Montgomery County (MD), a Linkages to Learning site, access to health care was greatly increased, reducing the percent of families who reported no health care access for their children from 53% to 10%, and those with no insurance coverage from 38% to 10%.

Neighborhood: Six programs reported lower violence rates and safer streets in their communities. A unique finding was the reduction in student mobility reported by the Polk Bros. Foundation Full Service School Initiative:

• While mobility patterns varied, average mobility declined from 1996 to 1999 at all three schools participating in the initiative to levels at or below the city-wide average. More children are in school for longer periods, creating more chances to learn and participate in programs.

It should be noted that most of the programs showed impacts on more than one outcome, reflecting both the design and comprehensiveness of the research and the program. For example, the Marshalltown, Iowa, Caring Connection program appeared to have impacts on both academic achievement and youth development. Children's Aid Society had an impact on school performance as well as parent involvement and community safety. Elizabeth Street in Los Angeles also improved school performance, lowered the dropout rate and brought parents into the school.

It is time for community schools to be recognized as an important component of the education reform movement. Most of these programs have goals not only to improve school performance, but also to change the lives of children and their families and reduce social barriers to learning. These initiatives recognize that the forces for upgrading the quality of education must be joined with the provision of strong supports.

Comparison with School Reform Movement

Community school providers may take some solace in the fact that few school reform models have been able to produce solid evidence of success. A recent publication, An Educators' Guide to School Reform, reviews research on 24 "whole-school," "comprehensive," or "school wide" approaches.¹ The concept of school wide reform clearly overlaps with community schools, and among the 24 models cited in the Guide, at least one appears here (School Development Program). This work was conducted by the American Institutes for Research (AIR) under contract with the major educational organizations.

Each of the 24 models is rated on Evidence of Positive Effects on Student Achievement, using a five-point scale that ranges from strong evidence of positive effects (4 or more studies showing impact on educational achievement) to a no research rating, indicating that no rigorous studies were available. Only 3 of the school reform models were rated strong, 5 promising, 6 marginal, 1 weak, and 8 had no research at all. The Guide stated, "In general, evidence of positive effects on student achievement - arguably the most important feature of any reform approach - is extremely limited. Even though many of the approaches have been in schools for years, only three provide strong evidence of positive effects on student achievement. As a result, educators often are considering schoolwide reform without vital information on which to make decisions. More rigorous evaluations are needed with broad dissemination of findings."