Guest Viewpoint: Test scores alone don't measure a school's success
By Dennis Littky

11/23/04 -- Every year under the No Child Left Behind system, you are given two or three words of evaluation for each school, like "making insufficient progress," or if you're lucky, "moderately" or "high performing." Think about it. Three words to summarize a school, and we believe it.

NCLB is so confusing that it's hard to comprehend all of the nuances and factors that go into the final score.

Standardized test designers never intended for them to be used as sole measures of a student's or school's performance. Yet, we give these tests enormous weight, when many factors like demographics or dropout rates could skew your scores.

So what is the alternative? If you want to really understand your schools, use many ways of assessing them.

The Met School, a public high school in Providence, R.I., looks at several measures of success. This school is the model for our education reform group, The Big Picture Company, which has started 24 schools in cities around the country, including Chicago, Detroit, Denver, Indianapolis, Sacramento, Oakland, and San Diego.

The Met has a diverse student population, and 82 percent qualify for the free or reduced-price federal lunch program. Students often enter the school with skills well below grade level, and many speak English as their second language. The Met scores above other schools with similar demographics on the state's mandatory math and English tests, even if these scores aren't always great.

"As we know, testing data strongly correlates to the student's socioeconomic status. But the data doesn't reflect that," says Stanley Goldstein, chairman of The Met School Board of Directors and former founding CEO of CVS Pharmacy.

At the Met School, we look at some of the following factors to get a clearer picture of how we are doing:

• Attendance data -- This tells you how engaging the school is. If students are involved in their work and are known by their teachers, they come to school.

• Dropout data -- Another measure of the attention paid to students, this data will often reflect two other important pieces: Is the work relevant? Are the students known well by their teachers?

If your school has a high dropout rate, your test scores might be higher because the weaker students are gone, whereas the school with the lower dropout rate (although better at retaining students) will most likely have lower test scores. Which do you want?

• College data -- The U.S. Department of Labor just released a study saying that seven out of 10 jobs today require a college degree. So shouldn't it be a goal for every high school graduate to be admitted to college or at least have the choice to enroll? What percentage enroll in college and how many stay and complete their degrees?

In its annual "State Report Card," the Rhode Island Department of Education reports the above data as well as the results of mandatory statewide testing and School Accountability for Learning and Teaching (SALT) Surveys, which are the culmination of thousands of intensive surveys of parents, students, and teachers at each school.
SALT should be a model for other states. The following are examples of some of the other data Rhode Island collects that could help you learn more about your school:

• Teacher availability -- Surveys pose the question to students, "Can you talk to your teachers about academic problems? About personal problems?"

The results in Rhode Island range from 8 percent to 50 percent for the first question and from 21 to 64 percent on the second. What does it mean if your school scores poorly on these measures?

• School safety -- Do you care about the perceived safety of your school? Students should be asked whether they have been bullied in the last year and whether they think illegal drugs are easy to get at school.

• Parent involvement -- You have an obligation to parents. Do you want to know if they feel involved in the school? How many times do teachers call them? How often do they come to school? How well does the teacher know their child?

• Class/ethnic diversity -- Is your school diverse? Rich? Poor? Predominantly African American, Caucasian, Latino, or Asian? Make sure the data is broken down by race and by class (those who qualify for federal free or reduced-price lunches and those who do not) so you can study the trends. Does the school look good because a small number of students take the SATs? Who takes Advanced Placement classes?

The Met School puts a lot of emphasis on these categories, so we pay close attention to our SALT Survey results. Last year, The Met ranked first in parent involvement, school climate, and quality of instruction. It had the lowest percentage of students who reported being bullied and the highest percentage who said they could talk to teachers about problems, both academic and personal.

This year, standardized test scores for Met students have gone up dramatically in math and reading. "It seems as though all of our data is more connected than we had originally thought," Goldstein says. "It looks like The Met's success in the SALT data predicted a strong academic culture which is now being reflected in state testing scores."

A school board should decide what pieces of data are important and insist that relevant information be collected. Creating a context with which to view the data is also crucial.
Know the skill levels of students as they come in to high school, so you will have a benchmark when they are tested later.

Otherwise, you could make changes in a school because you think students are making "insufficient progress" when, in fact, they entered high school with low skills and have greatly improved. Or, your scores could be skewed by having the poorest youths or a disproportionate number requiring special education.

Look at the research. Many studies set out the traits of "high-performing" high schools. These often include a common focus, high expectations, mutual respect between teachers and students, a personalized learning program for each student, authentic performance assessments, and in-depth learning. If you agree these criteria are important, have your superintendent look for ways to measure them.

Consider setting up teams to visit the school. There is nothing better than seeing the workings of a school for yourself to complement what the data tells you. Schools cannot change for your visit, so it will be real.

Pick teachers and kids at random. How do students feel about their work, their teachers, and their principal? Are the teachers happy to work there? Do they meet and work together? How are they growing?

Observe classrooms and walk around the school between classes. Are teachers and students talking to each other? Is there an attitude of respect in the school? Take a look at student work. Is there depth? Are students proud of their work?

Be clear about what measures you think are important and then get lots of data to find out if you are succeeding or not. Don't be irresponsible by depending on only one test for the answers.

Dennis Littky is director of The Met School and co-director of The Big Picture Company. He also is the author (with Samantha Grabelle) of the recently published book, The Big Picture: Education is Everyone's Business (ASCD Press).