Tuesday, March 2, 2010

On Databases, Indices, and Their Ilk

Squadratomagico has a very interesting post up about the so-called magic of databases, which should allow us to produce scholarship at a breakneck pace.**

My first reaction was: "Some of my documents are online, but until they invent a search engine that can read medieval handwriting and that can do fuzzy searches to account for obscure abbreviations and endless orthographic variants, no joy here for me."

And also: "Most of my best material is stuff I stumbled on while looking for something else entirely. This guy's method would never have uncovered this stuff."

But I was struck by one of the commenters on her thread, who pointed out that databases are rarely organized consistently, and one researcher's critical info is another database compiler's irrelevant chaff, to be left out or left unindexed. And I was reminded of my first foray into the archives. I had a really interesting idea for a dissertation topic – it had to do with gender and religion. Fine. And when I got to the archives, I knew which collection I wanted to look at, but it was really, really huge. Then, someone pointed me to the index.

Big mistake.

This is a multivolume handwritten "index" in which the indexer gave a one-line summary of the documents in each of the thousands of registers, indicating the folio number for each. Not detailed, but handy for directing you to what you were looking for.

Except it wasn't. See, this index had been created by some archivist working in the late 19th century, and he only chose to index the "historically important" documents. To him, this meant documents dealing with politics, finance, and land transfers. If I had gone from that index (and I did, for a while), I would have thought that there were no women anywhere in those registers. Also, no sex, violence, or petty squabbles between neighbors.

So, the original post author can write his articles based on his method. And I'll keep using the bibliographic databases that are available to me (how did I find books before WorldCat?). But I'm going to keep doing the work of scholarship the way I was taught. The databases are essential tools, but they can't be the sum of your research. Yes, it's hard work. Yes, it takes a long time. But what you get any other way isn't worth the half hour you put into it.

**Be sure to scroll down in the comments far enough to read Historiann's defense of the original author -- I did, and seriously redacted this post. But the central point is important nonetheless.


Comrade PhysioProf said...

Very interesting post. Two points:

(1) I find it fascinating that in order for you to study your medieval shit, not only do you have to master the medieval manuscripts themselves, but also a 19th century index.

(2) Most of my best material is stuff I stumbled on while looking for something else entirely.

This is how a lot of the most creative and groundbreaking science occurs. In fact, I was just discussing this with a colleague a few weeks ago, and we estimated that at least one third of Nobel Prize winning discoveries in the biomedical sciences occurred in this fortuitous way: the investigators weren't looking for what they ended up finding. The key, though, is to always be mentally prepared to resonate with the unexpected.

Historiann said...

CPP's comment is very wise: "The key . . . is to always be mentally prepared to resonate with the unexpected"

This might be TMI for non-historians, but wading through 19th C stuff is pretty common for most of us doing pre-19th C history. In many ways, I'm indebted to 19th C historians and antiquarians who thought it was a good use of their time and resources to publish scads of colonial town and court records, and even to index them. OTOH, as Notorious suggests, the prejudices and interests of an earlier era necessarily shaped what they chose to publish, and (very importantly) how they chose to index it.

But, we too are trapped in our own historical moment, and we'll be mocked and derided for our interests and obsessions by historians of the future, who won't believe what we thought was important, and what was irrelevant.

Happy researching!

Jo said...

Most of my best material is stuff I stumbled on while looking for something else entirely.

Thank you for this. It reminded me that I am, in fact, a researcher, no matter what that completely clueless program thought of me. (The one which gave me a vote of 'no confidence' re: whether I would finish a dissertation.)

I did this. I did this from the earliest age I can imagine, and I *still* do this. I get interested in a topic and start digging. I find some random incidental thing on the footnote on page xx and start digging about that.

It is what I do. I think I'll get to it now, with or without institutional support.

(word verification is remarkably latinate today: duledicu.)