War on Teachers I: GIGO

(The title is inspired by Historiann’s excellent post. Also a note: unlike most of the things I blog about, teaching is what I’ve done professionally for decades. I taught in universities, not schools, but the two aren’t totally unrelated.)

Every time you turn around, there’s a new front opened in the war on teachers. They don’t work hard enough. They get paid too much They’re not accountable. They can’t be fired. Their unions protect dead wood.

If we could just find the right stick to smash the cabal, the teachers would have to be good workers. Then, like good workers, they’d produce what they’re supposed to, which is good students.

So various fixes have been tried over the years. Student evaluations were big for a while. Who better than the consumers to point out when they’ve been sold a bill of goods? They’d identify the bad teachers in real time, and then we could start to get somewhere.

But it gradually became clear that if student feedback plays a role in teachers’ livelihoods, and the teachers have some control over that feedback, they’re going to make sure it leads to continued paychecks. Students want fun, easy classes with a good grade at the end, and there’s nothing but professional ethics to make teachers give them the grade they’ve actually earned. So grade inflation took off closely followed by the cries for accountability.

It’s obvious how to hold teachers accountable, right? They should be producing good students, so all you need is some quality control. Check out the product, and that’ll show whether the workers are doing their jobs. We know how to do quality control in education: we give exams and tests. So, it follows that testing the students will show how the teachers are doing.

No Child Left Behind was going to test every kid on the basics. That would show where the bad teachers and the bad schools were, and then the information would be used to help them get better. Or not. But either way, the problem would be solved.

Well, again, the part that was left out of the equation is that teachers do the actual nitty-gritty work of teaching, and if you’re going to make their livelihood depend on student tests they will — (this should be obvious, no?) — teach to the test. The tests weren’t measuring learning any more. They were replacing it.

That wasn’t working either.

People began to get desperate. Tests were the only real lever they had, and they weren’t working so they did the usual thing. They tried them some more. “No Child Left Behind” became the mad aunt in the attic that people try to forget, but her nephew “Race To The Top” is now messing about in the kitchen. Educrats began talking about doing testing right by using assessments and rubrics and Student Learning Outcomes.

To give a flavor of how these things function, I’ll go over another recent favorite: value-added testing. The idea is that students are tested at the beginning and end of, say, a year. Their scores are compared against their own past performance, and the improvement or lack of it measures how much “value” the teacher involved added during that year.

The LATimes recently did a series of reports that took several years’ worth of NCLB tests and used them to see how much value LA public school teachers were adding to their pupils.

Let’s not even worry about whether multiple choice tests are a good measure of how much anyone knows, although that issue has books written about it in its own right. NCLB tests were supposed to test “the basics,” reading and math. So, just to take a possibility at random, if you were a history, science, or art teacher (do they even have public school art teachers any more?) your students might not show much improvement in, say, math. But let’s not worry about that either. Let’s say the tests themselves provide some kind of useful information.

Let’s look only at whether value-added testing is valid, whether comparing a student’s grades against him- or herself tells you something about the teacher.

A look at the data shows problems that should be evident to anyone. For instance, the results indicated that math teachers who had Asian students in their classes were better teachers. That seems a bit odd. They also indicated that black teachers as a whole were much worse than white teachers who were worse than Asian teachers. The same disparity is not evident in those graduating from teaching colleges, so unless there’s something very peculiar about the Los Angeles school system, that’s a red flag for garbage-in, garbage-out. The articles didn’t mention that aspect of the results. Either the writers did not look at the data much, or they recognized the bogosity indicator enough not to mention it.

I won’t go into the statistical details of the study itself because it’s easier to understand the issues using an analogy. Take one huge problem right off the top: all of statistics depends on random sampling. The students of a given teacher are not a random sample. Students go to school near their neighborhoods, which only rarely reflect the general population. Using testing this way is like trying to gauge how good a municipal water system is by the look of the lawns on one short street. The two aren’t unrelated. If there’s no municipal water, they’ll all look bad. But even if every single lawn on that street is dead, it’s still not clear why. Foreclosure crisis? One broken water main? Major beetle grub infestation? In order to know whether the municipal system as a whole has a problem, it’s necessary to see a random sample of the whole city. If some lawns are okay and some aren’t, it might be the water supply. Or it might not. There are dozens of factors that go into lawns besides water.

There’s a creeping malaise that testing isn’t enough to create good schools. There are calls to ratchet up the pressure. Associate test outcomes with merit pay (and its unmentionable sibling, lower pay for teachers with students with brown lawns who don’t test as well). Associate test outcomes with prizes, with perks, with parking spots. If nothing works, associate them with being fired.

Unfortunately, the teachers’ unions halt progress at this point. They bleat about contracts and how hard teachers work. They talk about the resources needed for education. But they’re just teachers, so they’re only trying to protect their turf, obviously. They do everything they can to prevent the operation of an efficient system where the workers have to produce or get out.

There’s one last hope for fixing the system. Because of a some antiquated laws there’s no way to get rid of unions at public schools, so the solution is to dump the whole mess and start over with charter schools. Charter schools avoid unions, can operate with the lean efficiency of a business, reward the best, fire the worst, and, at last, produce those good students who are the whole point.

Except that they don’t. No matter what “Waiting for Superman” says. That film makes a good story out of some facts so selected that it does not suggest a desire to tell the truth. Diane Ravitch goes over the main points, so I’ll just give a few selected quotes.

Some fact-checking is in order, and the place to start is with the film’s quiet acknowledgment that only one in five charter schools is able to get the “amazing results” that it celebrates. … [T]he CREDO study … evaluated student progress on math tests in half the nation’s five thousand charter schools and concluded that 17 percent were superior to a matched traditional public school; 37 percent were worse than the public school; and the remaining 46 percent had academic gains no different from that of a similar public school. The proportion of charters that get amazing results is far smaller than 17 percent. [So not even 20%, or “one in five.”] …

While blasting the teachers’ unions, [Guggenheim] points to Finland as a nation whose educational system the US should emulate, not bothering to explain that it has a completely unionized teaching force. …

Guggenheim holds up Locke High School in Los Angeles, part of the Green Dot charter chain, as a success story …. With an infusion of $15 million of mostly private funding, Green Dot produced a safer, cleaner campus, but no more than tiny improvements in its students’ abysmal test scores. … Becoming a charter is no guarantee that a school serving a tough neighborhood will produce educational miracles. …

Another highly praised school that is featured in the film is the SEED charter boarding school in Washington, D.C. … It has remarkable rates of graduation and college acceptance. But SEED spends $35,000 per student, as compared to average current spending for public schools of about one third that amount. Is our society prepared to open boarding schools for tens of thousands of inner-city students and pay what it costs to copy the SEED model? Those who claim that better education for the neediest students won’t require more money cannot use SEED to support their argument.

So, what does it all mean? There’s a pattern here. Nothing works. And all of it starts from the assumption that learning is something bolted on in a not-very-serious factory called “school.” The teachers are the lineworkers, the students are the up-and-coming little Frankensteins before the bolts have been properly attached, and the unions are preventing the efficiency experts from moving the line along faster, cheaper, and better.

If the assumption is wrong, everything that flows from it is wrong. Garbage In, Garbage Out. The thing to do is step back and remember what’s involved in learning and teaching, which I’ll do in the next part. And then, the last part is to think about what that means on the ground for education.

If the assumptions and the goals are actually related to each other, people could stop wasting their time and money. Because what they’re doing now is like trying to launch rockets using boiled tapioca.

Posted to Acid Test, The Confluence, Corrente.