GregsOpinion.com

Gains in Houston Schools: How Real Are They?

As a student at Jefferson Davis High here, Rosa Arevelo seemed the "Texas miracle" in motion. After years of classroom drills, she passed the high school exam required for graduation on her first try. A program of college prep courses earned her the designation "Texas scholar."

At the University of Houston, though, Ms. Arevelo discovered the distance between what Texas public schools called success and what she needed to know. Trained to write five-paragraph "persuasive essays" for the state exam, she was stumped by her first writing assignment. She failed the college entrance exam in math twice, even with a year of remedial algebra. At 19, she gave up and went to trade school.

Set aside for the moment that single cases do not make for a trend. The example stated above is, nevertheless, a case in point of the fallacy that testing is an exact science that will dictate a precise measurement of aptitude, knowledge, or much else.

The article itself gets more scientific later on ...

With its own exam to measure pupil achievement, Texas managed to show educational progress over the last decade on a scale rarely, if ever, achieved before. But as the state's paradigm for school accountability became law for the rest of the nation, the authenticity of Texas's accomplishments has become a major question in education policy.

The Stanford test provides a useful contrast to the state exam, at least for Houston. More than 75,000 students in grades 3 through 8 and grade 10 took the state exam as well as the Stanford test from 1999 to 2002. The Times analyzed performances on these tests, excluding students in special education, and had educational testing experts review the results. The data were obtained under the state's open records act by George Scott, president of the Tax Research Association of Houston and Harris County, a taxpayers group.

"I don't think there was a miracle," said Robert L. Linn, co-director of the Center for Research on Evaluation, Standards and Student Testing at the University of Colorado, who reviewed the calculations. "There were some good positive results, but not extraordinary results like TAAS seemed to show."

The modest improvements in Houston have implications for the national debate. "If you anticipate that you can have the gains shown on TAAS ? and that's what No Child Left Behind would be requiring in many states ? that's not going to be likely to happen, based on this," Dr. Linn said.

The Times analysis of performance on the Stanford Achievement Test and the Texas exam shows this:

Houston students improved from 1999 to 2002 in most grades, but at only a fraction of the rate portrayed by the state exam. Using a widely employed statistical measure that allows different kinds of tests to be compared called effect size, the gains in the average scores on the Stanford test were about a third of the average gain in the TAAS scores.

Even students with the poorest skills posted high scores on the Texas test. In reading, a passing score of 70 on the test was the equivalent to scores below the 30th percentile in national ranking on the Stanford test in every grade. In 10th grade, passing the state exam was equivalent to the fifth percentile in the national ranking.

While the Houston gains on the Stanford test in some grades were large enough to be considered significant in educational testing, the city was not making much headway when compared with national averages. Some 57 percent of Houston students who took the math test in 1999 and 2002, and 51 percent of those who took the reading test, saw their standing relative to children around the country either fall or remain the same.

On the Stanford tests, the average reading scores for Houston students of all races in grades 9 through 11 have actually dropped since 1999. By contrast, the reading scores for 10th graders on the Texas exam ? the only high school grade in which the state test is given ? showed a large gain over the same period.

The achievement gap between whites and minorities, which Houston authorities have argued has nearly disappeared on the Texas exam, remains huge on the Stanford test. The ranking of the average white student was 36 points higher than that of the average black student in 1999 and fell slightly, to 34 points, in 2002.

"This says that the progress on TAAS is probably overstated, possibly by quite a margin," said Daniel Koretz of the Harvard School of Education, who also reviewed The Times's analysis, "And when all is said and done, Houston looks average or below average."

The methodology is here. About all I can add at this point is a shrug of the shoulders and a wry: "Color me shocked, why don't ya!" One problem with reform efforts is that when you reform nothing (effectively), and spend more money than needed to accomplish this net gain of zero, all while increasing the reach and scope of the federal government in education ... have you really done a world of good here? I obviously argue the negative on that point.

Detractors who might wish to point out that the case of the TAAS is distinct from my quarrels with NCLB might recall first that it was Ed. Sec. Rod Paige's "Houston Miracle" that fed into NCLB. Now that the miracle has been exposed as an accounting error at Sharpstown High and flawed testing to give the appearance of increased productivity in learning, I can't wait to see the response to this.

Reminds me ... would it just kill the Chron to ever dig up a real story in their own backyard???