A Qualified Defense of Standardized Testing in Higher Education

The New York Times recently ran a story about a Bush administration commission examining instituting standardized testing in higher education. The response from academe was predictable. Coturnix catalogued a few of them in his last teaching carnival (scroll about halfway down). Academic bloggers immediately point out that the No Child Left Behind act was disastrous for primary and secondary education, and that this looks simply like an attempt to do the same thing for colleges and universities. Many people also thought this was a clampdown on academic freedom and a part of the Republican war on science. This is all probably true.

But people also insisted that you cannot use standardized tests to measure critical thinking. Daniel at A Concerned Scientist says "We already know that standardized testing is very biased, and a poor indicator of meaningful learning and critical thinking, doing anything BUT contributing to learning and skill-building." Zandperl at Modern Science says "Someone needs to explain to him that all that standardized tests actually determine is whether (1) the student comes from a rich family, (2) the teacher was teaching to the test, and (3) the students are capable of memorizing."

Wait a minute. I teach a course on critical thinking almost every semester. I use a lot of tests in that course, more than any other I teach. These tests are standardized: every student gets the same one, and I use the same kinds of tests every semester. This is not something weird that I do: my tests are like those of almost every other critical thinking teacher in the English speaking world. My tests are not based on memorization. They measure whether the student is able to apply the skills we have been practicing to a novel situation. Nor do I teach to the test. Instead, I designed the test around what I teach. My tests are not direct measures of class background either. Students' scores do probably correlate with socioeconomic status, but this is only because high SES students are more likely to have the study skills that will enable them to learn the skills I am teaching. I am not bragging when I make these claims for my tests. I use the same textbooks and tests that everyone else does.

In other words, it is simply bullshit to say that you can’t test critical thinking. The disingenuousness of this claim comes out especially when people try to argue that standardized testing isn’t necessary because it is already done. Somehow critical thinking is both something that you can’t test and we are already testing for.

Moreover, those of us who teach critical thinking would benefit a lot from formally coordinating our tests. It would make it a lot easier both to identify best practices inculcating the skills we already teach and to determine which skills best transfer to nonacademic situations. (My personal hobby horse is the need to teach argument from authority as something other than a simple fallacy and to explain how to evaluate authorities.)

The UK offers standardized critical thinking A-level (advanced level) tests to students in the final two years of high school. One result of this seems to be a large amount of high quality material on teaching and testing critical thinking coming out of the UK, including some good organizations and textbooks. This leads me to think that a well executed program for evaluating critical thinking at the national level would be a good thing for teachers of critical thinking, and philosophers in general. It would be a chance to get sorely needed serious thinking about epistemology deeper into the standard curriculum.

This is not to say that I trust the current administration to deliver a well executed program of testing, driven by serious thinking about epistemology. Experience suggests that they will do the reverse. But this hardly means we can dismiss standardized testing for critical thinking out of hand.

Update: I removed a link from the sentence on good organizations in the UK when examination revealed that the organization I linked to was neither in the UK, nor very good.

Rifle Jesus

Pharyngula linked to this. I have no idea where it came from or what it means.

Becoming a hack

So I'm thinking about switching careers and becoming a hack. Churning out large quantities of formulaic prose purely for profit seems entertaining.

Given my current skill set, I should probably be a non-fiction hack, producing the sort of rewritten encyclopedia entries Isaac Asimov wrote in the middle period of his life. I've already got a good sense of the major tropes in the hack nonfiction formula. First there's the title, which should be something like

Magnesium: the Secret History of the Element that Created the Modern World.

Your topic is boring; to make it interesting, relate it to everything else on earth. An important variation of this in the environmental literature is based on using your boring thing to explain human history

The Yellow Turnip of the Patriarchy: How the Domestication of the Rutabaga brought the Downfall of Humanity.

The key to writing that book would be to argue that by domesticated the rutabaga, one takes an unprecedented level of control over one's environment that humanity simply doesn't deserve. The other important tactic I'll need to learn to write hack nonfiction is putting all the explication in direct quotes.

To learn more about the global rutabaga trade, I went to Professor Amelia Kindlybuttocks, handsome woman with a warm smile and a strange green growth on her forehead. "By putting the explication in the mouth of an expert," she explained, "the writer casts himself in the role of a student, thus getting the sympathy of the reader. The expert character cannot be too distracting, though. Otherwise the reader will lose the narrative thread. For instance, what the hell is this thing on my forehead? How does it explain the way rutabagas work as commodities?"

I'm fairly sure I could write thousands of words every day in this vein. The question is, what should I write about?

UPDATE: I completely forgot the best nonfiction hack trope, the Totally Bogus Personal Revealation (TBPR). This is actually a trick used by a popular nature writer I really like, Michael Pollan. Even in his hands, though, this rhetorical move is a sin. It goes something like this:

After leaving Professor Kindlybuttock's office, I drove to Pasta Bende, the world's largest magnesium mining operation. As I looked over the network of conveyer belts delivering precious magnesium to the hungry masses, I realized a profound truth: The nation that controls magnesium controls the world.

But you didn't have this revelation halfway through writing the book, now, did you. In fact, this point was your whole reason for writing the book, and the centerpiece of your book proposal. Of course, it is uncouth in popular nonfiction to simply have a thesis, so you have to have these artificial stories of personal revelations. These stories can get positively pernicious too, for instance when Bjorn Lomborg presents himself as an earnest environmentalist who in the course of writing his book just happened to come to believe in every major talking point put forward by big business.

What is preventing civil war in Iraq?

Not, it seems, U.S. troops. Supporters of the occupation have been saying that we need to keep our troops in Iraq to prevent the country from plunging into civil war. But during the recent violence, our troops have kept a low profile. The Sunnis have been drawn back to the negotiating table partially by diplomatic pressure, but mostly it seems by the simple realization that civil war would really suck. Really suck as in this picture from the New York Times article on the recent violence.
The Times' caption: "In Baquba, a father helped his wounded son Sunday after gunmen fired on boys playing soccer, killing two."

It's going to take a lot of talking to avert civil war, but its clear that the occupation, the nightime raids, and the secret prisons are at best superfluous in healing the country.

All bow before Heather Havrilesky

Of course, we can't learn from our mistakes if we don't recognize them as mistakes in the first place. If we see our mistakes as odd mixtures of circumstance and unfortunate luck, if we see them as twists of fate, or as insignificant side effects of a greater good, or worse yet, if we embrace our mistakes as beautiful creations filled with accidental grace that were simply interpreted by onlookers or by the media as mistakes, then we won't learn anything at all. That means we'll be doomed to repeat history, which means that big hair and boy bands and "Three's Company" will come back to haunt us, but instead of recognizing them as the colossal mistakes that they were, we'll thoughtlessly embrace them, putting the free world at great risk. That's right, Gavin MacLeod could rise to power once again, if we don't straighten up and fly right. --This week's "I Like to Watch"

Such wisdom from a T.V. review column.

The Quaker Menace

The administration and its allies are only trying to fight terrorism, except of course when they are fighting those who take a principled stand against violence altogether. This is something I've been noticing more and more of: the right is targeting not just people who might be sympathetic to Islamic causes (because of, say, their last name) and not just people who are opposed to the current war, but people who criticize militarism and violence altogether.

The Bush defense department has created a agency called Counterintelligence Field Activity (CIFA) which is designed to gather information on threats to the US military and US military contractors. The agency has a system called TALON ("Threat and Local Observation Notice") which gathers unverified reports on possible threats. (Is it just me, or were they really stretching to get an acronym that spelled "talon"?) Among the threats to national security is a Quaker meeting house in Lake Worth, Fla. The meeting lent their house out to an organization called the Truth Project, which tries to counter military recruiting of high school students. A member of the Quaker meeting attended the Truth Project meeting to learn more and found that their activities were very much in keeping with (Quaker) principles."

The administration's defense of this program seems to be that the information gathered by TALON is supposed to include a lot of false positives that analysts will weed out later. According to a pentagon spokesman the data collected by TALON "are unfiltered dots of information about perceived threats. An analyst will look at that information. And what we are trying to do is connect the dots before the next major attack." But this is only an attempt to raise the bar on what constituted domestic spying. Now it is ok for the government to keep records on you, so long as they don't use those records improperly. This is bullshit. The government shouldn't be keeping a file on anyone unless they can show the person to be a threat. The government shouldn't be spying on us "just in case." (For more on the Bush administration spying on its political enemies, see the resources Majikthise catalogued here)

In other news, David Horowitz believes
that a professor of peace studies at a Quaker university named Caroline Higgins is one of the one hundred worst professors in America. The criteria for being on the list are purely political. The academics aren't being judged on their scholarship--otherwise how could Chomsky be on the list? Nor do they seem to be judged by their teaching. At least, no effort is made to judge classroom performance, even in a Ratemyprofessor.com sort of way. No, Higgins' crime seems to be that she teaches courses with names like Methods of Peacemaking. Clearly this woman is dangerous. The last thing we want is for people to learn how to resolve conflicts without bloodshed, and perhaps even develop skills for harmonious living.

Now the neat thing about Horowitz enemies list is that you can vote on who you think is the biggest threat to the established order. Michael Bérubé has been campaigning quite successfully to raise his rating. Caroline Higgins meanwhile is languishing at the bottom of the list. I took the time to vote for her, assuming she'd recognize it as a compliment.

An intense sutta

I commend to you, the internet, this site which contains English translations of large swaths of the Pali Canon, the oldest Buddhist suttas, generally thought to be truest to the teachings of the Buddha's original order. The site began in 1993 as a BBS offering resources for Buddhist practitioners. Most of the translations I have seen so far are in fluid, idiomatic English, which is a breath of fresh air for me, since my library mostly stocks the old Pali Text Society translations, which were written in a deliberately anachronistic, King James style. I especially like the work of Andrew Olendzki, for instance this translation of 522 to 526 of the Theragāthā.

But what I really wanted to share is this story, from the Therigāthā of a nun who is pursued by an attractive young man, who asks her not to go forth into the contemplative life. (Scriptural spoilers follow, you may want to read the original first). The young man's opening line is fairly blunt.

You are young & not bad-looking,
what need do you have for going forth?
Throw off your ochre robe —
Come, let's delight in the flowering grove.

Well, "Not bad looking" isn't much of a come on line, but the young man goes on to praise her beauty quite extensively, especially her eyes. Sadly for our hopeful paramour, this is one of those texts that remind you that Gotama Buddha's middle path is substantially more ascetic that Aristotle's mean. Why are you so enamored of my body, the nun asks

What do you assume of any essence,
here in this cemetery grower, filled with corpses,
this body destined to break up?
What do you see when you look at me,
you who are out of your mind?'

Her equation on the composite nature of the body as the source of its impermanence and unworthiness is almost Platonic. But when she pulls away from the flux of the body, she takes a path far from Plato’s

Knowing the unattractiveness
of things compounded,
my mind cleaves to nothing at all.

A genuine lover is undeterred, so the young man continues to woo her, offering her riches and again praising her eyes. So he gets this response

Plucking out her lovely eye,
with mind unattached
she felt no regret.

'Here, take this eye. It's yours.'

Our symbiotes, ourselves

A couple months ago Steve sent me this short, enjoyable piece about the history of wheat from The Economist (Motto: "Oooh, Look At Me, I Read The Economist!"). The article gives you a portion of the conventional story of the role of agriculture in history: a happy tale of rising yields, rising standards of living, and increasing sustainable populations. It is also a story of how we worked ourselves into the tiny interstices of our environment, reworking the genomes of plants to turn them into symbiotes. Common wheat is a hexaploidal plant, bearing six copies of each chromosome, an exuberance due to the origin of common wheat as a hybrid of a diploid grass and tetraploid emmer or durum wheat. Emmer wheat apparently received its superabundance in the wild, though, and from emmer comes all domesticated tetraploidal wheat. The result of all of this crossing and recrossing, doubling and redoubling, is a plant that cannot reproduce without us, whose seed pod is shaped by human needs and not the function of dispersal. For our part, we could exist without wheat, but not in our current numbers. Hence, symbiosis. Because the conventional story of wheat is so uplifting—comedic in the older sense of the term—it gives us a happy, optimistic moral: we should embrace new genetic technologies and not worry about population growth. Everything is always getting better.

There is a counter narrative to the conventional happy picture of agriculture. (In the link, the counter narrative is told by Richard Manning, but it has many incarnations.) As we worked ourselves into the small parts of nature, we rolled over the large parts of nature. Manning describes wheat growing as catastrophe agriculture. Wheat is a plant that thrives on land that has cleared and flooded. When we grow wheat we have to remove the existing ecosystem, leaving a blank, nutrient rich slate for our symbiotes to spread. The domestication of wheat meant the clearing of the forest. This transformation of the surface of the earth by agriculture grounds the counter-narrative of agriculture, a story where human kind falls out of balance with nature and with itself, thus bringing about war, government, and inequality. Agriculture, although probably invented by women, was also a big step forward for the patriarchy. While this counter narrative is often told by those who oppose new genetic technologies and worry about population growth, the story itself has little in the way of a moral. Like all stories of humanity’s fall, it really just says humanity is shit outta luck. Everything is always getting worse.

I’m glad I have both these articles now. I have been assigning Manning’s version of the counter narrative for some time now, as if students actually knew the common story, which they do not. I now have two easy-to-read articles to assign together.

What I don’t have, sadly, is the truth. We need to tell a moral story of our tenure as a species here on earth, a story that can orient and vivify us. Darwin has blessed us with a tale of life on earth as a whole. There is a grandeur in this view of life, as well as a large dose of the truth, so I share a picture book version of it with my children. But our species is a minor character in the last act of this story. We need a smaller story to explain ourselves to ourselves.

I don’t like either the common story or the counter narrative I described above. They are starting points though, because they tell the story of our species together with the story of our symbiotes, and we would be fools to ignore how our successes and failures are bound up with the fate of the life forms we have surrounded ourselves with. (I am also convinced that the story of our species must include the story of dogs, and am fascinated by the possibility that the domestication of dogs began before our branch of the genus homo appeared.) Our symbiotes, ourselves.

Aloof professors complain about student email

The NY Times has an article up that quotes a bunch of professors complaining about "inappropriate" student email, where "inappropriate" means "not showing enough deference" “asking questions the professor thinks is stupid” or “providing feedback on the instructors performance.”

Most of the emails teachers complain about are notes I’d be happy to get. An email that says "I think you're covering the material too fast” is listed with messages that “go to far.” That’s silly. I’d love to know if any of my students thought I was going to fast, or too slow, or whatever.

Another professor complains about this email: "Should I buy a binder or a subject notebook? Since I'm a freshman, I'm not sure how to shop for school supplies. Would you let me know your recommendations? Thank you!" Apparently the teacher believed this was inappropriate because it wasn’t about the content of the course. But the kid is a new to college for God’s sake, and one of the things we are supposed to do with new students is teach them study skills. (The correct answer, by the way, is to get a large spiral notebook with pouches that you can put class handouts in and perforated pages so that you can turn in handwritten work without those irritating dangling fringe things.)

I can think of a few problems with student expectations of email, but most are simply extensions of other common student mistakes. No, I’m not going to be able to read 50 rough drafts the night before the paper is due. No, I’m not going to do your work for you: you will have to think this through yourself. None of these things really bother me: they certainly aren’t newsworthy. They just come with the territory.

I do have one question: how come students never use the subject line? Most student email says “(no subject.)” Is this because students are used to IMing?

Models of teaching

The literature on college teaching says we should move from being a "sage on the stage" to being a "guide on the side." What other models of teaching are there in higher education?

the bore at the door
the pest on your desk
the creep in your seat
the maniac in your backpack
the lodestone on your cell phone.

As a follower of Socrates, I prefer the "pest on your desk" mode of teaching. How do you teach? Do you know of other good teaching models that rhyme?

Reader services

Looking at sitemeter, I see that a lot of people are aparantly coming here looking for information that I do not provide. In an effort to serve you better, I will begin to try to answer some of your questions, or at least refer you to people who can.

1. Christian Women Help with Orgasms. Oh honey, stop thinking about Jesus and just relax. Jesus will understand.

2. Prophet Mohammed Cartons. If you drop an "o" from cartoon, you wind up finding a post I wrote, which linked to Fafblog's post about the Prophet Mohammed appearing on milk cartons. The information you want is really here, at wikipedia. It includes, for now, a small reproduction of the cartoons, along with this interesting paragraph on the role of pictures of the Prophet in Islam:

The Qur'an, Islam's holiest book, condemns idolatry, but has no direct condemnations of pictorial art. Direct prohibitions of pictorial art, or any depiction of sacred figures, are found in certain hadiths, or recorded oral traditions.

Views regarding pictorial representation within several religious communities have varied from group to group, and from time to time. Among Muslims, the Shi'a Muslims have been generally tolerant of pictorial representation of human figures including Muhammad. Indeed a fatwa exists given by Ali al-Sistani, the Shi'a marja of Iraq, stating that it is permissible to make pictures of Muhammad, if done with the highest respect. [58] Sunni Muslims are considered less tolerant. However, the Sunni Ottomans, the last dynasty to claim the caliphate, were not only tolerant but even patrons of miniaturist art, some of which depicted Muhammad. These depictions usually show Muhammad's face covered with a veil or as a featureless void emanating light (depicted as flames). Pictorial surveys of Muhammad can be found on the internet.[59][60][61]

Recent searches the brought people here

9-11 Bombing for kids

Christian Women help with orgasms

buddhism is evangelism practiced

really ugly fish

Female Monkey Clitoris

John Rawls Kristen Hersh

Salon on why they are holding back

Salon editor Joan Walsh has a piece up replying to some criticisms of their release of more Abu Ghraib pictures. Most of it deals with the total red herring issue of whether they should have also published the offensive Prophet Mohammed cartoons. (Isn't it enough that they linked to them?)

She also has a word to day about why they didn't publish everything they have. Fifty one words, to be precise.

But we have also rejected the notion of a quick and dirty dump of the contents to the Web. Some significant portion of the documents we possess does not appear to relate at all to prisoner abuse at Abu Ghraib, and we can see no public interest served by publishing it.

There is also a general tone to the article which says, "We need to research this material more carefully--Be sure it is what it looks like, find out what we can about context--before putting it out there."

Fair enough. But I still have questions. First, who else have they shared these with? The ACLU sued for access to these. Do they have a copy now? (I get the impression that all of this fits on a single DVD.) How about the International Criminal Court? Second, can you confirm Seymour Hersh's description of the contents of some of these videos.

No, Madam Press Secretary, I didn't really think you would answer those questions. I just thought they needed to be asked.

Sliver

Caroline: I wanna hear the Gramma Take Me Home song!

Molly: What do you want?

I put in With the Lights Out disc 2, which has an acoustic demo of Sliver, which Caroline has become enamored of.

Caroline: It’s a song about a boy who doesn’t want to go to his gramma’s house, and he says gramma take me home, and then he falls asleep in his mother’s arms and wakes up in her tummy.

Me: Actually, this version doesn’t have the nice ending verse I was telling her about, [singing] “after dinner I had ice cream/I fell asleep and watched TV/I woke up in my mothers arms.”

Caroline [Singing]: gramma take me home, gramma take me home, gramma take me home.

Me: Actually, lets listen to the rock out version of this song (*)

Caroline [Singing]: gramma take me home, gramma take me home, gramma take me home.

Getting the electric version takes a little time, because I have to make a copy of it first. We have a rule to never put original copies where the kids can get them.

Later:

Kurt Cobain: GRAMMA TAKE ME HOME, GRAMMA TAKE ME HOME, GRAMMA TAKE ME HOME

Caroline: Why is he yelling?

Me: He’s upset.

Caroline: It’s not nice to yell at your gramma.


____
(*) in our house any song with electric guitars is called a “rock out song”