Machine Learning and Legal Tech: Three Cheers for Humans

Written by Ross Guberman on March 23, 2021

Shape Left bg
Shape Right bg

I shared two data-mined facts about briefs the other day. If you’re skittish about whether artificial intelligence threatens the lawyers of tomorrow, the reaction I got should reassure you.

Here were the facts I shared:

  1. The term “Id.” appears half as often in the briefs of elite lawyers (according to published rankings) as it does in randomly selected briefs.

  2. The term “v.” appears 25 percent less often in the briefs of elite lawyers as it does in randomly selected briefs.

I didn’t comment on these facts. I just presented them. Almost right away, though, my avid Twitter followers and others duked it out. The responses ranged from rage to “Who cares?” to intellectual excitement. Many seemed both confused and curious.

What did it all mean?

Three Cheers for Humans

Cue humans. For three reasons.

1. Understanding which data points matter.

Id.” appears less often in briefs by nationally renowned advocates than it does in my collection of randomly selected briefs. But so does the word “green.”

Focusing on that second data point is a waste of time. The lower rate for “green” simply means that my random collection just so happened to include more cases about green lights or green widgets.

2. Understanding what data points mean.

Many followers thought that the lower rate of “Id.” meant that elite lawyers simply used other ways to signal repeat references to the same authority. Others surmised that perhaps more elite lawyers write Supreme Court briefs, which use “Ibid” rather than “Id.” (I had checked for that, but would algorithms know to do so?) Still others suggested that elite lawyers might be so arrogant that they don’t cite authorities at all.

All these conclusions are rational. But it’s because of all the other data I have, and my decades of experience analyzing and writing books about briefs, that I know the likeliest reason for the differing rates: On average, the best brief-writers are less likely than the average lawyer to quote or paraphrase several points or facts in a row from the same authority. They also cite fewer unique cases on average even though they don’t necessarily write shorter briefs.

Those insights, individually and in combination, are far more valuable than is the number-crunching alone.

3. Putting data-backed interpretations in context.

My dry data points on “Id.” and “v.” rates elicited surprisingly strong reactions. Some lawyers seemed defensive, finding the data as controlling and judgmental rather than inspiring or at least provocative. Was I implying that they should write more like “elite” lawyers? I wasn’t saying anything of the sort, but I needed to make that clear.

On the other side of the spectrum, some lawyers took the data not just seriously but literally, interpreting the numbers to mean that they should never use “Id.” and only rarely cite any cases at all. It’s unreasonable to expect busy professionals to understand the implications of differing rates of terms on average.

Perhaps it also takes a human to empathize with our fellow humans’ thought processes. After seeing a slew of replies, I realized that I had led many followers to believe that the differences in usage rates were per brief rather than per 1,000 words, as was the case. Those followers were using impeccable logic to draw sensible conclusions from accurate data, but because of assumptions they were making, their conclusions weren’t as helpful as they could have been.

That was my fault, I thought, so I cleared things up. But would even the most intricate computer algorithm have had that flash of empathy?

Legal tech without lawyers? Don’t count on it.

Posted in
Ross Guberman

Ross Guberman

Ross Guberman is the president of Legal Writing Pro LLC and the founder of BriefCatch LLC. From Alaska and Hawaii to Paris and Hong Kong, Ross has conducted thousands of workshops on three continents for prominent law firms, judges, agencies, corporations, and associations. His workshops are among the highest rated in the world of professional legal education.

Leave a Comment