Computer-graded essays

Slashdot pointed me to this New York Times article about computerized grading of student essays. Apparently, every high school junior in the state of Indiana is now required to take an essay test that is graded by a computer.

With the increasing number of mandates to test student writing, “there’s a certain inevitability to computerized essay grading,” said Stan Jones, Indiana’s commissioner of higher education. Indiana’s computerized essay scoring, he said, will reduce by half the cost of administering a pencil-and-paper test and will free teachers from distributing, collecting and, above all, grading thousands of test booklets.

Goodness knows, we can’t have teachers wasting their time grading student work! Next thing we’ll be asking them to teach them something.

Most of the comments on the Slashdot thread about this article seem to focus on the geek fantasy of writing a nonsensical essay that will fool the computer. I actually think anyone who can do that should be given an automatic “A” in English, because they will have probably learned more about the English language than most of us will ever know. More perceptive commenters suggest that the fact that the computers’ grading is indistinguishable from that of humans is more likely a negative reflection on the human graders than an endorsement of computer grading.

The bigger problem is this one: Computers cannot yet “understand” language. As Steven Pinker points out in The Language Instinct, the potential ambiguities in any sentence make the process quite difficult: humans can quickly determine the appropriate interpretation through context; computers are unable to understand context, and therefore they flounder, and so have difficulty understanding the meaning of texts. The sentence “Time flies like an arrow,” for example, can be interpreted in five different ways . Here are just a couple of ways:

When timing houseflies, time them in the same manner in which you time arrows
A type of fly, a “time fly,” enjoys the company of a particular arrow

Longer sentences have even more possible parses — an average of 27 in sentences ranging from 1 to 25 words long, according to a Carnegie-Mellon study. The problem becomes exponentially more difficult as the complexity of the topic and the sentences increases.

Computers may be able to effectively imitate human graders because grading is inherently arbitrary. One person’s masterpiece is another’s mindless drivel. There is no doubt that computers certainly can help students become better writers, but when computers replace the role of human teachers in the writing classroom, the question inevitably arises: why are we bothering to teach the humans how to write, anyway?

This entry was posted in General. Bookmark the permalink.

One Response to Computer-graded essays

  1. Pingback: Word Munger » Happy birthday, Word Munger!

Comments are closed.