The Art of Test-making

Making out good tests isn’t easy. A variety of flawed techniques will rear their ugly heads early in one’s teaching career, ugly heads that I have seen up close and personal with my own two eyes. The dangers include:

• Vague questions that students will argue over
• Questions that don’t really test anything you’ve taught
• Impossibly arcane and tricky questions
• Tests where the entire class scores about the same
• Tests where the hoped-for bell-shaped curve looks more like a cardiogram readout
• Tests that take too long to take
• Tests that don’t take long enough
• Tests that take too long to grade
• Tests that take too long to type out
• Tests that use up too much paper
• Tests that are difficult to decipher, both in content and in form

The list goes on. How can the beginning teacher avoid these land mines? Of course, experience will teach you a lot, but as the old saying goes, the fool learns from his mistakes; the wise man learns from the mistakes of others. Since I’ve already made all of the mistakes listed above, let me save you from folly.

My Sorry Story

Early in my teaching career this is how a typical test-making-out night transpired.

I grabbed all the important books, notes, and files that I thought were germane and headed home from school. Back then, I always made my tests out at home, which was my first mistake. Why? Because I never grabbed enough books, notes, and files.

Then I rolled some typing paper into the typewriter (this was 1975, mind you) and typed

Name _________________________

I was on my way.

Hm, what would be my first group of questions? I liked matching because it allowed for recognition rather than recall...plus I had excelled in the process-of-elimination aspect of matching in my test-taking days. The problem was, what to match with what? Suddenly, on the night I was making out the test, I didn’t know what I’d taught them. This wasn’t good.

This sudden realization betrayed an element of something that most beginning teachers have to deal with: undefined objectives. It wasn’t until test time that I really was forced to consider what I wanted my students to learn.

This sounds terrible -- I mean, a teacher that doesn’t know what he’s trying to teach is unfit for the profession, right?

Wrong. Please allow me a paragraph or two to defend the teacher who doesn’t know exactly what he’s doing. Somebody needs to do it...believe me, the critics of public education won’t do it.

Teachers who aren’t quite able to articulate just what their teaching objectives are, quite simply, rookies. How can any beginning teacher know exactly what they’re doing? Most beginners operate one of two ways: they either mimic what their college profs taught them or they follow the school’s existing curriculum guide to a tee. Well, there is a third option: the free-wheeling charismatic soul who plans little and shoots from the lip. It doesn’t matter which of the three one is, if he is a beginner, he probably is doing a lot of driving in the fog, if you follow my metaphor.

The cure is simple: experience. Nothing, and I mean nothing, takes the place of four or five years of teaching to help clarify what a teacher is doing.

Besides, there are a lot of people, myself one of them, who think “exploratory teaching” has its major benefits. Most of my best ideas come during the course of a unit, not at the beginning. Many come during a detour or even following false lead. I know this about myself: if I limit myself to my pre-articulated goals and objectives, or last year’s test for that matter, I limit my teaching in a major way.

So all you readers who have felt bad for wondering, just what the heck did I teach those kids, take heart. You’re okay. You’re a good person. You’re smart. You just haven’t been at the job long enough. Plus...you haven’t (up till now) read this article. Because, as I said earlier, you’re in the process of learning from my mistakes.

Okay, detour over. We’re back at where I was on test-making-out night: staring at a white sheet of paper wondering what in the world I was going to quiz my students over. At this point, I usually began to consider what had gone on in class. I tried to remember the discussions we’d had. I knew that none of my four sections of sophomore English had covered the exact same terrain. I worried that I hadn’t even given one group the vocab or another group all the same stories to read. Should I just ask questions as if I was sure they all had covered the same material?

Yes. That’s what I learned to do. It was easy to tell, on test day, if they hadn’t. I’d just wait for the processional to begin: Student A, B, C, and D would march to my desk and say, “We never read that story.” And, after one or two reliable ones said it, I’d announce, “Hey, class, you can all skip numbers xx through xx.” They’d be happy, I’d be, well, not happy, but at least at peace with myself (well, maybe not total peace) and all was well.

So one thing I learned in those early days was not to be afraid to alter the test on test day. This made making out the tests ever so much less stressful.

Another thing I learned as I was making out tests went right to the heart of those objectives I wasn’t so sure about. As I was making out the test, I began to really see what it was I wished I had taught. Or taught better. This, rookie teachers, is a common emotion felt by the most experienced of veterans. It often takes a sojourn through a unit with a real class in front of you to make you aware of what’s important. Those beautiful ideas, complete with stated goals and objectives, that you created prior to teaching the unit, get whittled down to reality by a roomful (or 5 roomfuls) of kids.

And here’s where I learned a second important test-building trick: “learning” tests. What’s that? It’s a test that’s half-test, half-worksheet. I would allow a certain amount of time during the test-taking with notes or I would give fresh information at the beginning of the test period. I remember one time in particular when I had neglected to teach the concepts of “static” and “dynamic” characters. To remedy, I simply explained them before class and then used questions that required the students to apply these concepts to the characters we’d just read.

By serendipity -- or my own stupidity and sloth -- I had stumbled onto one of the major hidden beauties of test day. It’s the day the kids are most open to learning. When you’re giving notes on day one of the unit, three weeks prior to a test, kids are thinking, “Whoop-dee-dee. I can study these later.” When you’re giving information on the day of the test and telling kids how to use it to enhance their scores, they’re listening. I now make all my tests have some degree of new learning in them. The trick is not to test them over the new learning. You simply use it to apply to the material they were supposed to have learned. It really works.

From this, I learned another secret: lots of tests. Lots of tests means lots of grades which translates to less grades variability. (There is a handy term statistics people use regarding this -- something to do with standard deviation, but I can't remember exactly which term it is. Anyway, you get my drift, I'm sure.)

Related to this, more tests allows me to grade written work differently... and not as harshly. Kids won’t argue as much about a low test score as a low composition grade. They also won’t accuse you of grading favoritism.

And I learned other test tricks. For instance, I learned...

• how to write questions that allowed for some “play” in the outcome, a little fuzz factor.
• how to be fair in grading.
• what type of questions needed most careful wording.
• how to proportion tests so that everyone has a shot at passing
• how to write analytical questions
• to always give explicit pretest reviews
• how to write easy-to-grade tests.
• when it’s appropriate to have kids grade tests for you.
• how to quickly grade essay and short answer questions.

All these things I learned by trial and error. Had I a document such as the one you’re reading, I probably could’ve avoided some of the error part.

I’m telling you my story for several reasons. For one thing, I want you, the beginning teacher, to realize you’re not alone in the fog. And I also want you to know that you can navigate in it, if you’re creative. Furthermore, I think beginning teachers need to see the connection between tests and objectives. Tests really are nothing more than your teaching objectives in question form. Despite what I said earlier about the value of “exploratory” teaching, we do need to have objectives in front of us most of the time. It sharpens us as instructors and students as learners.

I’ve heard teachers moan about “just teaching for the test.” The sentiment is wrong: teaching for the test is simply teaching towards the goals you have.

Finally, I want you to see from my early experience -- and the fact that I survived -- that you don’t have to be perfect, nor do your tests. Teaching is a growing experience. Making out tests will make you a better teacher.