Our next fun word for the A-Z Blog Challenge is “data-driven.” In education, everything now needs to be “data driven.” The DATA we derive from testing should DRIVE our instruction.
Here’s the problem with data DRIVING our instruction – where did the data come from and why are we putting it behind the wheel? Students sit in cold desks in sterile classrooms for hours filling in bubbles on district produced benchmarks that promise to tell us exactly what each student knows and doesn’t know so we can better prepare our students for the state test.
When I worked at TCS, we sent the benchmarks back to district officials who broke down the tests question by question. We could see each answer choice that each student picked. The district mandated that we take students who missed each question, put them in small groups based on the question missed, and then work with them on that individual standard until they learned it. According to the district, my class should never be an entire class, just a bunch of small groups working on individual standards all around the room.
That’s a nice fantasy. During a “coaching” session from my curriculum specialist, I noticed some major problems. Krystle, Hannah and Ryan, three of the most brilliant students in my classes, didn’t pass the assessment. These are kids who are destined to someday win a Nobel Prize, who never turn in work that earns less than an A.
The fact that they didn't pass was a major red flag to me. “There’s something seriously wrong with this test,” I said.
“Oh no, that’s not possible,” she responded.
“But it is. How can three of our most high-achieving students fail it?” I showed her the results.
She inspected it. “Maybe they aren’t good test takers?” she offered lamely.
“Okay,” I said, “if that were true, which it isn’t. Then the test doesn’t measure what they know, but it measures their test-taking ability. And we’re not testing for that, are we?”
She brightly said, “Why don’t we just look at some of the questions they missed?” We did and saw that all three of them missed question 15. “So clearly, this standard is an area they are weak on!” she announced triumphantly.
I’m not sure why she was triumphant. “No, something’s wrong with the question,” I pointed out. “For one, they all missed it, and we covered that concept in class three weeks ago. They all did well. Furthermore, 95 percent of the students missed it. One who answered correctly is the lowest student in my class, and I can tell you he doesn’t understand the concept. The question is bad. And overall, it looks like the test is bad.”
She started babbling about how she needed to look into it, how the district ordered the tests from a certain company and sometimes there were mistakes, blah blah blah. At this point, I knew it wasn’t even worth bringing up a more glaring issue of a test creator who thought student mastery of a standard could be assessed based on a single question. The entire benchmark was only 20 questions long and claimed to test 15 standards.
The curriculum coach tried to get back to analyzing the test with me, but I insisted there was nothing more to learn. “I’ll be happy to give the students a new, improved benchmark and study the results from there,” I said firmly.
I found out a week later that she had lied to me about the test. It hadn’t been created by a test company that had made a mistake. She wrote the test, and when I with other teachers pointed out that the results didn’t add up, she chose to put the blame on a phantom test company.
State standardized test are created by experts, who spend years and millions of dollars to create these exams. District and state funding is tied to how well the students do on the tests. In a desperate attempt to raise scores, districts everywhere are throwing together practice tests that they promise will produce data that will drive teacher instruction toward student success. The trouble is, the data produced has the effect of a schizophrenic driving you home. He doesn’t even know who HE is, let alone where you want to go. We’d do better by students, to put teachers behind the wheel.