Humans have it easy. If we want to know whether two words alliterate, we can just listen and decide for ourselves.  It’s poetry, so if it sounds good, it is good.  Computers don’t have that option.  We have to tell it which sounds are equivalent and which are different.  The basics are taken care of by the pronouncing dictionary, but we still have to deal with the edge cases.

In Anglo-Saxon verse, Jun Terasawa is the authority. That’s the scheme I’ll start with. B, D, F, G, L, M, N, P, R, T, and W are in classes by themselves; they don’t alliterate with anything but themselves. The complicated classes are: {G, even when pronounced like Y}; {C, whether pronounced as CH or K}; {S}; {SC}; {SP}; {ST}; and {A,E,I,O,U,Y,H}.

The pronouncing dictionary takes care of G and C.  The letter “S” is different from the three di-consonants it helps form in Anglo-Saxon. (but read on!)  The vowels have a couple of interesting features. Not every case of a word beginning with a vowel alliterates. This came up in the first version of the program: It found the line, “Aragorn and Éomer and Imrahil rode” and said that was a good alliterative line. I put it to the company at Thursday Nights at the Green Dragon, all of whom agreed it was not.  This is because the thing that’s really alliterative isn’t the vowel, it’s the glottal stop when the previous word ends with a vowel, too. (Terasawa, §2.1.) That’s how the “H” finds its way into that group. In a sense, H plays the same role as the glottal stop. We know this is a complicated issue because the English still have problems with words beginning in “H”.

Now off to the computer. We know a syllable has the stress when its vowel has a “1” after it. So when the computer finds a “1”, we can tell the computer to back up to the previous consonant and that’s the sound on which we alliterate – almost. Some stressed syllables begin with two consonants, and we want to alliterate on the first of them. Like “blue-embroidered”: using the naive rule rule would say the consonants to check are L and R, and the computer would return “no”. But those of us with ears to hear would call that an alliteration on B. So we build those in as special cases to test first.  The di-consonants are: BL BR CL CR DR DW FL FR FY GL GR KL KN KR PL PR SC SK SL SM SN SP SQ ST SW TR TW.  When it sees one of these cases, the computer has to back up two consonants to find the beginning of the syllable.

I’m breaking some of Terasawa’s rules since we’re speaking Modern English — I think phrases like “second story” alliterate now, so “S” and “ST” are in the same equivalence class. “S” and “SH” are still different, so “SH” isn’t in the list of di-consonants.

Works Cited

Lerner, Alan Jay and Frederick Lowe. My Fair Lady. New York, 1956.

Terasawa, Jun. Old English Metre: An Introduction. University of Toronto Press, 2011.

Prominent features in Anglo-Saxon poetry

The cool thing about Old English alliterative verse is that the rhythms can shift around freely. The hardest thing about teaching a computer to recognize it is that the rhythms can shift around freely. There’s no pattern.

Well, that’s not strictly correct. There is an enormous literature devoted to finding patterns. Daniel O’Donnell wrote an excellent short introduction to the topic, which I used to get started. Here’s something he says that I believe to be universally true: a line of alliterative verse is comprised of two half-lines. Each half-line is built around two important stressed syllables. After that, things start to fall apart. The patterns of stressed and unstressed syllables have been classified according to many systems, the most durable of which was devised by yet another German with an infinite attention span, this one named Sievers. He identified five rhythmic patterns: A, B, C, D1, D2, and E. See the problem? Thomas Cable came up with a better way to think of it [1], and concluded that D2 was actually a form of E, but there are still six things in this set of five.

This is a phenomenon I recognize from my day job: it’s a sign that the analyst is imposing a structure on the system, not observing a structure that is really there. And it gets worse. Some lines have five stresses, others have three, weak positions in the line come and go as the poet pleases, the caesura might be a full stop or it might not be discernable with the ear. The exceptions to any metrical system pile up in epicycles upon epicycles until the readers throw up their hands and go plant potatoes. (It was sunny and 60 degrees today, so that’s what I did.)

Here’s where it turned out to be a good thing that I held off on this post to take a class in Anglo-Saxon at Signum University. Nelson Goering is one of the professors teaching the class. He wrote a very detailed description of alliterative scansion, accessible to the educated layman (as Scientific American used to say). And his paper makes the same curious turn of thought, as if he’s classifying an existing fauna, not dictating to it. I was about to write an essay on the theme of “You’re all full of crap.” But now that I’ve listened to his lecture on the topic, it’s clear that they all know they’re full of it, but conceal the fact for purposes of creating scholarly literature. Lectures are great — they give the speaker a lot more leeway to tell embarrassing truths.

Here’s another thing I think is universally true: Poets are not natural rule-followers. They write what sounds good. You know who nature’s rule-followers are? Scientists. We typically write humorous verse, which is the one genre of poetry where meter must be followed exactly. Here’s a poem by James Clerk Maxwell, to show what I mean.

Tolkien was a poet, not a scientist. So when Treebeard says, “Learn now the lore of living creatures: First name the Four, the free peoples”, I’d have to say he’s writing alliterative verse. But there are five stresses in these lines (bold), and the alliteration is on 1, 3, & 4. The rules say the extra stress goes in the second half, not the first. But I don’t care. I’m going to teach the machine to recognize things like this, and forget the rules.

[1] Cable, Thomas. “Metrical Simplicity and Sievers’ Five Types.” Studies in Philology 69.3 (1972): 280-288.

Attention conservation notice: Preparing a text for automated poetry detection is half spell-checking. That part may be kind of dull; I’m writing it down anyway because this blog is how I remember things now.

Cleanup

To do automated text processing, you need a text. Some people, like the indefatigable Sparrow, type the whole text by hand. I am more defatigable than in-, so I needed another way. The Internet Archive has a text version of The Lord of the Rings, left over from the days of freedom before the Enclosure of the Internet. I
suspect that the lawyers have let it survive because the quality of the scan is so poor. I started with that.

I did most of my work on the Unix command line. My constant companion is a program called “aspell“. Feed it a text file, and it returns a list of all the words it didn’t recognize. A scanner makes predictable errors, such as reading a “u” as “ii” or an “h” as “li”. Those were the most common misspellings. They are easy enough to encode in a stream-editor input file. So I cleaned up all of those that I could. Some were harder, like distinguishing between “ore” and “orc”. The computer will never get that straight, so I did those by hand. All ambiguous cases got changed to “orc”, and then I went to the Moria chapters and turned back the two or three actual mentions of “ore”. (I have no idea how hard this task would be with a book I didn’t know so well, but I may soon find out.)

[ETA: The Guardian has a hilarious article saying that I didn’t get the really fun scanner error.]

The next thing to do was restore the hyphenated words at the ends of lines. That’s easy to do with a little Perl script — check lines that end in hyphens; delete the hyphen and newline, and see if the combination passes spell-check.

Now comes the fun part. LotR is full of proper names, archaic words, British
spellings, and invented languages. The spell-checker will never get those
right. So I had to work my way through the aspell output, picking out by hand the
briticisms and other things that were correct but unknown to the software. This took up my spare time for about a week. The result is a long list of words that are
correctly spelled, despite what the spell-checker thinks. So the process was:

1. Run aspell on the text;
2. Use the comm command to compare the output to the list of correct words. Remove all the words in the output that are correct in this context;
3. Figure out what the corrections might be on the remainder.  If they’re not wrong, add them to the “correct” list.
4. Repeat until the remainder is empty.

The list of correct words is an interesting document in its own right. You never know when that might come in handy, so I’ve published it on the Humanities Commons Core.

We now have a text of the book. Text is not all we need, though. We need sounds.

Phonetics

Next, we have to find the poetry. Alliteration is easy to identify, in theory. We’re  going to look for words that begin with the same sound. Or, rather, words that have the same sound at the beginning of the accented syllable. This is done with a “pronouncing dictionary”. I used the free one from Carnegie-Mellon University. (These were invented so voice-mail systems could read you a printed text over the phone, but they can be used for Good, too.) For each word in the English language, it has a line with the spelling of the word its phonetic expansion:

VOLUNTARY  V AA1 L AH0 N T EH2 R IY0
ASSISTANT  AH0 S IH1 S T AH0 N T
POSTMEN	 P OW1 S T M EH N

The list of phonemes are on the welcome page of the dictionary, but they’re easy to figure out.  Vowels have stresses; the numbers are the amount of stress each vowel gets. Zero means unstressed.  One means stressed. Subsidiary stresses have higher numbers.  Straight out of the box, the dictionary has over 130,000 words in it, but of course that’s not enough.  I spent the next couple of weeks making dictionary entries for all the LotR-words I collected in the last step. The supplementary dictionary is also on line at the Humanities Commons.  Surprisingly, the CMU dictionary contains all of the phonemes I needed except one. The supplement includes “KH” for words like “Erech” and “Grishnakh”.

This concludes the boring part. Next, I plunge into the quagmire of Anglo-Saxon scansion.

One doesn’t have to get very deep into studying Middle-earth to notice Anglo-Saxon alliterative verse. It’s right there on the surface, but an American teenager probably won’t get that it’s a structured verse-form. (At least, I didn’t. I thought it was another kind of vers libre.) I don’t know what fraction of Tolkien fans end up fascinated by this mostly-forgotten poetic form but I’m one of them. 

After one gets the hang of its rhythm, the really interesting part starts: alliterative lines start to show up in all kinds of places in Tolkien.  The first one I noticed was when the Riders of Rohan came to the Pelennor Fields:
‘In dark Mindolluin’s sides / they dimly echoed.’

Years later, I was listening to the Prancing Pony Podcast and heard (line breaks and caesurae added):

Then the warriors of Nargothrond went forth, and…
Tall and terrible / on that day looked Turin,
And the heart of the host / was upheld, as
He rode on the right hand / of Orodreth.[1]
Silmarillion, chapter 21, p. 239.

I started wondering how many of those buried alliterative lines there are in The Lord of the Rings. If there are a lot, they’re almost certainly carrying meaning. Could this be one of the techniques JRRT used to give a feel for where the story was in space and time? Maybe we’ll see lots of them in Rohan (even apart from the poems), and almost none in Lothlorien. If this works, it might be actual scholarship.

I raised one objection with myself: isn’t this just an Anglo-Saxon version of the “found poetry” that was all the rage in the 1960s? My English teachers inflicted that on us and I hated it. Without “intentionality” (as real scholars call it) it can’t be poetry, was my main objection.  I think this isn’t the case, for two reasons. First, nothing Tolkien wrote was unintentional. If alliterative lines are there, he left them there for us to find. Second, the poems they find always seemed to be free verse. If the poetry-finders had ever found a sonnet, or even a limerick, I’d have thought much better of them. Here, I’m looking for a strict verse form.

Granted, it’s possible to accidentally create a formal alliterative line. Tom Shippey pointed out that Steve Earle did it in a country song. Earle wrote it as a couplet:

When your Subaru is over and your Honda’s history,
I’ll be burnin’ down the back roads, just my baby and me…

“Sweet Little ‘66”, lines 22-23

but that second line scans perfectly. This isn’t the same thing as found poetry, because Earle was intentionally writing verse, and liked the alliteration. It’s the mirror image of what Cædmon wrote :

Þa middan-geard  / mann-cynnes weard

“Cædmon’s Hymn”, line 7

Which rhymes, but the rhyme is a poetic flourish, not part of the structure.

In keeping with proper Idiosophical practice, I decided to give the job to a computer. [2] Telling what happened is too long for one blog post, so I’m making this into a series.  Part 2 will be the delights of getting a text in shape. Part 3 will be a scream of frustration at the scholarship of Old English poetic meter. Part 4 will be algorithms, and Part 5 will be the results.

Teaser: the first thing the computer found was something I’d been staring at for a generation without seeing: “Bilbo Baggins / of Bag End” .

[1] This is how I learned the correct pronunciation of “Orodreth”.

[2] That was almost a year ago – I could have done it by hand by now.