Home
kids

Evaluation of Community Schools:
An Early Look

Working Draft
Joy Dryfoos, author



The first question usually asked about community schools is "what are they?" and the second is "do they work?" This brief describes what a community school looks like, summarizes what we know about the impact of community schools on a range of results, and highlights three recent evaluations of community school initiatives. At this stage we know that community school initiatives are beginning to produce positive results, and increasing numbers of principals and teachers are testifying to their value in helping to improve student learning and strengthen families and communities.

What does a community school look like?

'Community school' is an inclusive term, encompassing a growing number of school-community initiatives that feature both common themes and differing approaches. The names of the initiatives suggest some of the varying attributes: Caring Communities, Beacons, Bridges to Success, University-Assisted Schools, Healthy Start, Communities in Schools, School of the 21st Century and many others. Some are broad in scope, promoting widespread replication or adaptation, while others are single entities. Programs are being initiated at the national level (e.g., Children's Aid Society, School of the 21st Century), state level (e.g., New Jersey School-Based Youth Program), local level (e.g., Polk Bros. Foundation Full Services Schools Initiative in Chicago) and in individual schools (Molly Stark Community School, Bennington, Vermont). Community schools also vary in their goals: Some specifically aim to improve academic achievement while others focus primarily on health and behavioral outcomes or enhanced family functioning.

Although the Coalition recognizes that each community school is unique, it has developed a general description of a well-developed community school:


Vision of a Community School
A community school, operating in a public school building, is open to students, families and the community before, during, and after school, seven days a week, all year long. It is jointly operated through a partnership between the school system and one or more community agencies. Families, youth, principals, teachers and neighborhood residents help design and implement activities that promote high educational achievement and positive youth development. The school is oriented toward the community, encouraging student learning through community service and service learning. A before and after-school learning component encourages students to build on their classroom experiences, expand their horizons, contribute to their communities, and have fun. A family support center helps families with child rearing, employment, housing, immigration, and other services. Medical, dental, and mental health services are readily available. College faculty and students, business people, youth workers, neighbors, and family members come to support and bolster what schools are working hard to accomplish - ensuring young people's academic, interpersonal, and career success. Ideally, a full-time community school coordinator works in partnership with the principal. This person is responsible for the delivery of an array of supports provided by local agency partners and participates on the management team for the school. Over time, most community schools consciously integrate activities in several areas to achieve the desired results: quality education; positive youth development; family support; family and community engagement in decision-making; and community development.

How do we know if community schools are effective?

Policy makers and practitioners want to know what kinds of positive changes community schools can make. Before addressing this question, we first ask, "Given our vision, what indicators of success should we be looking for?"

We believe that community schools should be seen as vehicles for education reform; therefore, improved learning and achievement must be a long-term measure of the effectiveness of this growing movement. In addition to test scores, learning and achievement related indicators include rates of attendance, promotion, graduation, suspension and expulsion for example.

It is important to note that community schools are designed to affect not only educational outcomes but other outcomes as well. Such outcomes include improved social behavior and healthy youth development; better family functioning and parental involvement; enhanced school and community climate; and access to support services. These outcomes have value in and of themselves, in addition to affecting educational outcomes.

New Jersey School-Based Youth Services Program: A State Program
The New Jersey School-Based Youth Services Program (SBYSP), developed by the New Jersey Department of Human Resources in 1987, was the first major state program that gave grants to community agencies to link education and human services, health, and employment systems. The 'one-stop' program has been initiated by schools and community agency partners in 30 school districts (at least one in each county). Each site offers a range of services including crisis intervention, counseling, health services, drug and alcohol abuse counseling, employment services, summer job development, and recreation.

The Academy for Educational Development conducted a 3-year longitudinal evaluation of the SBYSP which compared survey responses and school data for students who took advantage of SBYSP services and activities with those who did not, while statistically controlling for important background variables, such as family stress, access to support from family and other adults, and participation in positive youth activities. The evaluation findings showed that SBYSP was clearly reaching the most vulnerable students in the six outcome-study schools and had been able to make important differences in the lives of these students.

Overall, analyses of students' responses to the baseline and follow-up surveys, controlling for baseline levels of behavior, family stress, family and other adult support, and participation in youth activities, showed positive change on 39 of the 45 outcomes studied in the evaluation, and statistically significant positive change on 14 of the 45 outcomes. The behavioral and attitudinal outcomes that showed the desired change at statistically significant levels included educational aspirations; academic credits earned; trouble sleeping; feelings of unhappiness, sadness or depression; worrying 'too much'; feelings of anger and destructiveness; suicidal thoughts; use of contraceptives to prevent pregnancy; use of condoms to prevent STDs; smoking; and engaging in deliberate property damage, as well as on boys' involvement in hitting others with intent to hurt.


The data summarized here indicate that community school initiatives are moving in the right direction across many different indicators of success, giving us cause for optimism. The story is not complete however. First, it should be noted that the programs we identified cover a broad continuum, from highly developed community schools that have been in existence for a decade or longer to schools that are just beginning to open their doors and offer expanded opportunities such as after-school activities. Second, the quality of the studies varied enormously, from evaluations that relied on very small non-representative samples to those that were based on carefully designed management information systems and control groups. At best, evaluation is difficult, expensive, and long-term. Only a few programs can produce what would pass as 'scientific' results. Many others can offer 'preliminary findings', early returns on long-term projects.

All around the country, researchers are working to develop ways to do a better job of documenting the effects of these programs. The constraints are many when one is trying to track events in an experimental program offered in a setting (public school) that does not lend itself to experimental conditions. (see the full paper for a discussion of the challenges.) With these complexities in mind, Lisbeth Schorr and Daniel Yankelovich (Los Angeles Times, Feb. 16, 2000) have argued for moving ahead with social programs and not getting bogged down in methodological warfare: "Evaluating complex social programs is not like testing a new drug. The interventions needed to rescue inner-city schools, strengthen families, and rebuild neighborhoods are not stable chemicals manufactured and administered in standardized doses. Promising social programs are sprawling efforts with multiple components requiring constant mid-course corrections, the active involvement of committed human beings, and flexible adaptation to local circumstances." What is most important is that there is a growing body of evidence that community schools are beginning to demonstrate positive effects on students, families, and communities. The preliminary data summarized here suggest that many of these models have the capacity to produce multiple impacts that include, and go beyond the expectations of traditional education reform.


Polk Bros. Foundation Full Services School Initiative: a Local Program
The goal of the Polk Bros. Foundation's Full Service School Initiative is to improve the physical and psychological well-being of children in three elementary or middle schools in order to make a positive impact on their school-related behavior and academic achievement. Support services are brought into the school that meet the needs and desires of families and strengthen the relationships between parents and school personnel. Each partnership is required to set up a governance body and hire a full time coordinator to oversee the operations.

The three schools and their partners are Brentano Elementary/Logan Square Neighborhood Association, Marquette Elementary/Metropolitan Family Services and Riis Elementary/Youth Guidance. The schools are open after school and in the evening. Each has a different set of programs including parent involvement, recreation, school remediation, and tutoring.

Emerging evidence from an evaluation of the Full Service Schools Initiative conducted by the Chapin Hall Center for Children at the University of Chicago is promising:

  • Reading scores improved at rates exceeding the city-wide average at all three schools. Improvement in reading has been among the toughest challenges facing Chicago public schools.
  • By the first half of 1999 all three schools were actively acquiring resources from over 25 outside organizations each.
  • Parents surveyed reported an increase in the number of adults in after-school programs who could be trusted to help their child with a serious problem. As parents come to see the school as a friend of the family and a safe haven, they are more likely to support the school in maintaining high expectations for learning and appropriate behavior. The climate of the school improves.
  • Teachers surveyed reported an increase in the number of adults in after-school programs who know children in the school well as individuals.

Sources of Information

This is a first cut at compiling documentation on the impact of community schools. Many evaluations are currently underway, and within the next several years we should have access to much more information. The full paper contains some 78 references. An examination of those citations will confirm that the 'literature' about community schools is mostly 'hidden'. Compiling the information involves extracting annual reports and unpublished documents from researchers and program administrators throughout the country. Web sites (identified where known) are a great benefit because one can download reports that are difficult to obtain in hard copy and have not been (and probably never will be) published.

We were able to obtain information on 49 school-community programs that have produced recent evaluation reports or data on results. No two initiatives are alike nor are the research protocols used. Of the 49, six are after-school initiatives such as LA's Best. Some research reports, for example on the large-scale California Healthy StartInitiative, aggregate findings for hundreds of schools. Other programs, such as Communities in Schools, have submitted reports at state, community, and local site levels (counted as one report). Other evaluation efforts focus exclusively on one school such as the study of the Broad Acres Elementary Linkages to Learning School in Montgomery County, Maryland. Thus, the count of reports is rough and is intended to serve as an estimate of the large quantity of information that is becoming available. Three examples of evaluations from the full report are provided here in order to illustrate the scope of efforts underway.

All of the reports present findings on one or more outcomes, such as improved academic achievement, change in student behavior, or increased parental involvement. In 46 of the reports some positive changes were noted. In three of the reports, no positive changes were reported - that is, students in the schools with the programs did no better than those in comparison schools, or alternatively, students in the program did not improve their performances or behaviors over time.

The School of the 21st Century: A National Model
The School of the 21st Century (21C) is a model for school-based child care and family support services that transforms the traditional school into a year-round, multi-service center providing high-quality, accessible services from early morning to early evening. The ultimate goal of the School of the 21st Century is to help provide affordable, accessible and high-quality services for all families, regardless of income level, to ensure the optimal development of children. While each 21C school varies according to local needs and resources, the model includes six core components: parent outreach and education; preschool-age programs; before-, after-school and vacation programs for school-age children; health education and services; networks and training for child care providers; and, information and referral services. Since its inception more than 600 schools in 17 states have implemented 21C programs.

Research indicates that the combination of services provided in the 21C model has strong benefits for children, parents and schools:

  • Children who participate in 21C beginning at age three start kindergarten more ready to learn as evidenced by their scores on kindergarten screening tests.
  • Additionally, children who participate in 21C for at least three years evidence higher scores in mathematics and reading achievement tests than children in a comparison, non-21C school.
  • 21C parents reported that they experience significantly less stress, as measured by the Parental Stress Index; they spend less money on child care, and they miss fewer days of work.
  • The addition of early childhood classes to the school has had a positive impact on teaching practices, with teachers in the primary grades incorporating the best aspects of early childhood classrooms.
  • The expanded services provided by 21C schools have improved their standing within the larger community, as evidenced by more positive public relations, the passage of significant bond issues, and a substantial reduction in school vandalism.

Key Results

Highlights of the compilation on outcomes are presented below. Outcomes include learning and achievement, improved social behavior and healthy youth development; better family functioning and parental involvement; and, enhanced community life.

Learning and Achievement

Achievement: Thirty-six of the 49 programs reported academic gains. These gains generally included improvements in reading and math test scores, looked at over a two- or three-year period. Many of the programs reporting academic gains were in elementary schools. In at least eight of the cases, the outcomes were not schoolwide. Rather, they were limited to students who received special services, such as case management, intensive mental health services, or extended day sessions.

However, there were some instances of school-wide improvements on academic measures:

  • Charles Drew Elementary School, a participating school in University of Pennsylvania's West Philadelphia Improvement Corp program, showed more improvement on the state's standardized reading and math tests than any other school in the state in 1999, an increase of 420 points.
  • At PS 5, a Children's Aid Society Community School in New York City, the percentage of children reading at grade level rose from 28% when they were ingrade 4 to 42% by the time they reached grade 6.
Attendance: Nineteen programs reported improvements in school attendance. Several reported lower dropout rates, one specifically among pregnant and parenting teens. Several mentioned higher teacher attendance rates, suggesting higher levels of satisfaction. Examples of positive results in this area:

  • A national evaluation of Communities in Schools found that about 70% of students with high absenteeism prior to participation in CIS improved their attendance.
  • At Lane Middle School, a Schools Uniting Neighborhoods sites in Portland OR, attendance increased from 85% to 91% over two years.
Suspensions: Eleven programs reported a reduction in suspensions. This may reflect changes in suspension policies, rather than changes in behaviors leading to suspensions. As schools transform into more child-centered institutions, they are likely to change practices regarding suspensions and expulsions as part of the change in school climate.

  • At the Woodrow Wilson Middle School in Des Moines, Iowa the rate of suspensions in 1995 was one-sixth the rate five years earlier.
  • At Lane Middle School in Portland, OR, suspensions declined from 50 to 15 over two years.

Social Behavior and Healthy Youth Development High-risk behaviors: Eleven programs reported reductions in rates of substance abuse, teen pregnancy, disruptive behavior in the classroom, or improvement in behavior in general. For example:

  • An evaluation based on 138 grantees in California's Healthy Start initiative found that students receiving Healthy Start services decreased their drug use.
  • The Blenheim School (Missouri Caring Communities site) reported a 40% decrease in disruptive behavioral incidents following the initiative of a system for referrals for clinical therapy.

Family Well-Being

Parent involvement. At least 12 of the programs reported increases in parent involvement. For example, at the Bryant School, a Caring Communities site in Missouri with an intensive family intervention program, volunteer hours increased from 43 in 1996 to 2,008 in 1998.

Family functioning. In many programs with a strong family focus, improved family functioning was reported.

  • Parents who received School of the 21st Century services were able to improve their child development practices, were less stressed, spent less money on child care, and missed fewer days work.
  • An evaluation of 138 grantees of California's Healthy Start initiative found that families reported improvement in filling basic needs such as housing, food and clothing, transportation, finances and employment.

Community Life

Access to services: Better access to health care, lower hospitalization rates, higher immunization rates, or access to dental care were reported at least once. After-school programs cited access to child care as a significant outcome.

  • As a result of putting full time school nurses in school sites in the Success Program in Des Moines Public Schools, 97% of the children were immunized. Dental screenings produced a 34% increase in improved oral hygiene and a 36% decrease in referrals for cavities.
  • At Broad Acres Elementary School in Montgomery County (MD), a Linkages to Learning site, access to health care was greatly increased, reducing the percent of families who reported no health care access for their children from 53% to 10%, and those with no insurance coverage from 38% to 10%.
Neighborhood: Six programs reported lower violence rates and safer streets in their communities. A unique finding was the reduction in student mobility reported by the Polk Bros. Foundation Full Service School Initiative:
  • While mobility patterns varied, average mobility declined from 1996 to 1999 at all three schools participating in the initiative to levels at or below the city-wide average. More children are in school for longer periods, creating more chances to learn and participate in programs.

It should be noted that most of the programs showed impacts on more than one outcome, reflecting both the design and comprehensiveness of the research and the program. For example, the Marshalltown, Iowa, Caring Connection program appeared to have impacts on both academic achievement and youth development. Children's Aid Society had an impact on school performance as well as parent involvement and community safety. Elizabeth Street in Los Angeles also improved school performance, lowered the dropout rate and brought parents into the school.

The full document contains reference to many more examples of ways in which community schools are having positive impacts on a range of outcomes. It is time for community schools to be recognized as an important component of the education reform movement. Most of these programs have goals not only to improve school performance, but also to change the lives of children and their families and reduce social barriers to learning. These initiatives recognize that the forces for upgrading the quality of education must be joined with the provision of strong supports.

Acknowledgements

The Coalition appreciates the generous support of the Carnegie Corporation, Charles Stewart Mott Foundation, Ewing Marion Kauffman Foundation, and Wallace-Reader's Digest Fund. We are grateful to Joy Dryfoos for writing this document and Laura Samberg for editing it.

>>> Home