If you are a Centennialist author, rule number one is you stay the hell away from numbers except to round them up until they show great big globs of zeroes. (Once rounded, they can be used for any nefarious activity.) If questioned about your numbers, you answer “No one can know exactly how many men … so we must use estimates.”Exception: if you are super-scholarly, prize-winning, unbelievably detail-oriented, you grab the first set of craptastic OR numbers you see and cite those as authoritative.
These are the advanced numeric arts of the Centennialist.
Joseph T. Glatthaar is an early middle-aged Centennialist being groomed by Gary Gallagher to walk in the shoes of himself, Sears, McPherson, and the old storytellers – Williams, Williams, Catton, etc. Up until recently, Glatthaar’s management of numbers has fit the mold. As recently as in General Lee’s Army (2008), he says (for example) that at Cedar Mountain Jackson “lost over 1,400 men,” while he inflicted casualties of “2,400 men.” I didn't bother to look up the citation he gave for these shiny, shaky jell-o confections.
Thomas Livermore attempted to correct the OR numbers at the turn of the century. It was a first try and it suffered many drawbacks but his book, Numbers and Losses, is useful and a big step up from the OR. Where Glatthaar puts Jackson’s casualties at 1,400 Livermore shows them to be 1,338 killed and wounded. Where Glatthaar puts Banks’ casualties at 2,400 Livermore counts 2,353, including “missing.” BTW, Livermore does not have “missing” figures for Jackson, so the careless researcher adding up killed, wounded, and missing would have apples (Jackson) and oranges (Banks).
Livermore shows numbers down to the last digit in an easily accessible reference work, amply noted and accessible to professors like Glatthaar, while Glatthaar chooses to display his jiggly gelatin dessert collection instead. Would you like whipped cream with that data? Now, General Lee’s Army is a book otherwise rich in statistical demographic detail (it even has decimal points!) – but the work is as indifferent to analysis in the matter of numbers and losses as any Centennial work you’ve ever seen. A house divided: care for one sort of number, contempt for others.
What good is a blog if we can’t conjecture? My suspicion is that Glatthaar designed the book and had graduate students (or other helpers) develop it while he supervised them. The parts in which he had been indoctrinated (master narrative, numbers and losses, major themes and ideas) reflect his personal involvement to ensure conformity with the canon; meanwhile, the non-battle numeric detail represents the work of others.
In General Lee, Glatthaar gives a lot of individual credit to his statistical helpers but the book smacks of a general collaboration, perhaps even a committee production, with JG presiding to keep the doctrinal bits on track.
This post is not about General Lee’s Army, but a few more notes are in order before moving on. I have harshly criticized Glatthaar in the past for his lack of historical sense and feeling. This is amply displayed in General Lee’s Army and it creates in the reader a jarring disconnect between the slovenliness of Glatthaar’s narrative shortcuts and the effort taken to generate his demographic information.
Look at this sentence: “Nearly half the soldiers who ultimately served in the Army of Northern Virginia enlisted in 1861, and another third joined in 1862.” As of when? Is this statement calibrated to an ANV status as of June, 1862, when the ANV was formed; or to an end of year 1862 status; or to an end of war status? When did the remaining one sixth of “the army” (one half plus one third = five sixths) enlist, in 1860? Or were they draftees?
Again and again, in General Lee’s Army, Glatthaar loses track of the context and his readers feel the loss. Over the years he has communicated to us that he does not give a fig about this or that distinction; suddenly, he’s dropped us in a sea figures, analysis of which requires painstaking distinctions.
As I said about another author, there is still good history work that can be done by doctrinaire, narrative-driven authors who lack historical sensibility. Basic research is among these.
Maybe I was wrong.
General Lee’s Army is packed with demographic research which is distilled, separated, and presented in a new book, Soldiering in the Army of Northern Virginia: A Statistical Portrait of the Troops Who Served under Robert E. Lee. This book represents methodological problems from beginning to end, some of which we’ll explore below. It bids to become a standard reference work and it plays straight (more or less) by documenting (in a hopscotching, find-the-snippet way) how it came to many of its figures.
What this book needed was an extensive methodological essay with a literature review relevant to the techniques selected, not the Easter egg hunt given us. After that, it needed to be reviewed by social scientists expert in statistical research methodologies who could comment on its many problems before this went to press.
Unfortunately, Soldiering, to the careless reader or hurried researcher, is going to be a “black box” of a book, a collection of handy outputs that can be sprinkled like bacon bits on whatever narrative salad is coming out of the kitchen. Never mind the ingredients…
The statistical basis for some of the material in Soldiering is actually better explained in General Lee’s Army, for instance:
The sample was designed by Dr. Kent Tedin, the former chairman of the Department of Political Science at the University of Houston. The sample consists of 600 soldiers who served in Lee’s army. Because there was no single list of names, we chose a stratified cluster sample. Each infantry, cavalry, and artillery unit that ever served in Lee’s army received a number. I then determined through army strength throughout the war that 81.8 percent of all troops were in the infantry, 11.3 percent were in the cavalry, and 6.9 percent were in the artillery. We then randomly selected fifty artillery batteries, fifty cavalry regiments, and seventy-five infantry regiments. We then randomly selected three names from each chosen battery and cavalry regiment and four from each infantry regiment. The sample consists of 150 artillerists, 150 cavalrymen, and 300 infantrymen. The artillery and cavalry samples are large enough to make them statistically significant. The infantry sample is much larger because of the proportion of infantrymen in Lee’s army.Soldiering glosses this with less detail. Both mention that Glatthaar researched 54 data categories per soldier over a period of years.
The questions begin immediately and there are no answers. This is the line but where is the staff? Are the NCOs proportionally represented or not? If the artillery and cavalry samples are just large enough to be “statistically significant,” don’t they become insignificant if there are gaps in the 54 data categories, especially if those gaps form a pattern? Why is a sample of 150 good enough for cavalry and infantry but 300 is needed for infantry? What is the margin of error on a sample like this? Does the sample take names from a set point in time? How does this selection represent soldiers across time during the whole war? Do the number of soldiers killed correspond proportionally to the numbers killed or are the survivors or dead (respectively) over- or underrepresented?
Every time Glatthaar notes an anomaly (highest average wealth among 1862 enlistees, for instance) the reader is left wondering whether the sample was adequate.
By referring to his friend Kent Tedin the way he does, Glatthaar both credits the man and personally distances himself from the methodology. This is a rum business. Glatthaar has to own the work product and he is clearly ill at ease down in the weeds. In General Lee’s Army, he says “…we chose a stratified cluster sample…” In Soldiering, he says “Kent had to conceive a more complicated process … In technical terms it is called 'a stratified cluster sample.'”
Stratified cluster sampling is a hybrid; there is cluster sampling and stratified sampling. Glatthaar’s commentary describes (partially) cluster sampling but not stratified cluster sampling.
Glatthaar mentions names were drawn from artillery batteries, cavalry regiments (I assume: maybe squadrons?), and infantry regiments. Assume these are the clusters. The battery, an infantry company sized unit, yields three names and an infantry regiment 8-10 times larger, yields four. The cavalry regiment at strength holds roughly half the men of an infantry regiment but yields three names to the infantry’s four. Furthermore, as these represent three types of unit what limits are there on comingling sample data? What are the strata? Are the strata auxiliary data points like age or slave-owning?
Among users of stratified cluster sampling there seems to be a need to do variance weighting and for that a number of software programs are available. This offsets the inaccuracies in cluster sampling. Glatthaar says he learned Stata in preparing this work. What did he use it for and what adjustments had to be made to the data using Stata and why?
I feel okay looking at a chart showing “Year of Birth” for artillerymen drawing on data from 150 artillerymen. I feel worse looking at a chart showing “Personal and Family Wealth by Region” based on just 600 lives. I feel awful reading a chart that shows 600 lives yielding “Killed and Wounded by Slaveholding Status.” I eventually expected to see a chart, "Wounded slaveholders further injured by lifting heavy objects in the presence of civilian women 27 years of age or older."
To assuage our anxieties about his methodology, Glatthaar has an immense number of footnotes complementing himself and his techniques for accuracy. Here are a two:
In the chi-square test for slaveholders, P=.0000 indicating an accuracy of more than 99%.Occasionally, he notes a result outside the acceptable limits for social science standards. But what does this gibberish mean?
In the chi-square test for killed and wounded, P=.0546 indicating an accuracy of 98.6%.
It seems to me that Tedin used Stata to run this test on this or that data in the book and then entered a jarring, conceptually incomplete end note showing the test result. How was the test run? No answer. Glatthaar says merely “there is a chi-square test.” There is. It exists. It shall continue to be. Strike the gong and let the references to this test begin!
Again, I want to stress Glatthaar’s unfitness for history, and it really shines through in this kind of naiveté:
If a scholar searches long enough, he or she will find evidence to justify virtually any contemporary attitude and buttress virtually any argument the scholar may pose, regardless of its representativeness. For that reason, valid statistics may break that scholarly logjam.And you’ll have the last word, of course based on this book.
This testament of faith follows an equally interesting statement:
In some instances, statistics vary slightly between this book [Soldiering] and that one [General Lee’s Army]. Since the publication of General Lee’s Army, I have come to realize that a few soldiers in the census were not the soldiers I was seeking, despite their having the right name and age. As a result, minor changes appear in this volume.And that’s as close to humility, circumspection, and respect for history as we get. Strike another gong.
***
P.S. The application of a stratified cluster design to Glatthaar’s project IMHO would normally have resulted in an entire randomly selected cluster (unit) being analyzed across strata (criteria), like wounds or wealth, rather than in applying successive strata to a pool of random selectees plucked out of a cluster.
For a comical collection of student comments on Dr. Tedin, see link.
If you want to do a little chi-square figuring have at it!
I reviewed General Lee's Army here, passing over the data elements.
Any of Glatthaar’s military histories will indulge your taste for goose egg pornography in a big way. Here’s a reviewer thrashing in ecstasy at the sight of page 134 of The March to the Sea and Beyond: “Sherman’s 60,000 troops encountered little resistance: They cut a wide swath through the heart of Georgia, and confiscated nearly 7,000 mules and horses, over 13,000 head of cattle, over 10,000,000 million pounds of grain, and nearly 11,000,000 million pounds of fodder in the process.” Can anybody reasonably trust this man with a chi-square and five decimal places?
