A (gentle) defense of an epistemic rule of thumb

Wendy the psycho therapist expressed doubt about an epistemic rule of thumb I suggested. The rule of thumb is: "When unorthodox methods lead to unorthodox conclusions, you should doubt the methods, rather than embracing the conclusion."

This is really a simple corollary to the skeptic’s principle, “extraordinary claims require extraordinary evidence,” with the elaboration that “extraordinary evidence” doesn’t mean unusual evidence, but evidence that is very hard to dispute.

Wendy worries, “What are you considering "unorthodox" and who/what is making this determination?” Context would help here. The rule came to my mind while I was reading a student paper on the death penalty. The student reported roughly that a doctor so-and-so using “new statistical methods” was able to show that seven lives were saved for every one person executed. My first thought was “Wow, that’s quite a claim, those had better be some good methods.” I suspect, though, that these new methods were simply invented for the purpose of justifying the death penalty and that there is no independent evidence for their reliability. In fact, this smacks so much of pseudo-science that I don’t even want to waste time investigating it.

So what am I counting as unorthodox? I am assuming a roughly scientific context, where there are some agreed upon methods that have yielded agreed upon results, and some other methods and results that are debated. I think most human knowledge can be well characterized this way, although the size of the agreeing communities may vary. My experience has been that good epistemology is actually profoundly conservative. At the epistemological level, I am practically Edmund Burke. We are born into believing communities, and when those believe systems are dysfunctional, we try to repair them with minimal changes, lest we find ourselves lost. Paradigm shifts are painful, and should be postponed, until it is clear that only the most ad hoc changes can keep the old paradigm afloat.

Wendy also said “IMHO experiences exist which cannot be easily or readily quantified, reproduced, expressed and/or explained and validity and reliability, orthodoxy and unorthodoxy are rendered moot.” I am aware of those experiences, too. But if expression is difficult, and reliability moot, what else is there to say?