Confounding Variables

I always find it amusing when politicians use newspaper articles in their ads to support their claims, especially when those claims are tied to numbers. Speaking as a journalism sinner, a good number of us went into journalism because we had a phobia regarding math. I’ve known great, award-winning journalists who couldn’t balance their checkbooks, so when I see ads like this one from Scott Walker, I must pause and ponder.

Truth be told, I got over some of my math phobia when a stats prof pulled a Jedi Mind Trick on me during my doctoral program. “Don’t worry about being afraid of math,” he told us. “Stats aren’t math.” It worked just enough to get me by and fall in love with data crunching.

Sadly, I think the phrase “stats aren’t math” has been taken to a whole other level in this gubernatorial campaign.

After getting his ass kicked by job-loss numbers for several months, Walker released unvetted federal data that showed the state not only didn’t lose jobs, but instead gained about 30,000 of them since he took office. Democrat Tom Barrett rightly pointed out that this appears to be a case of data shopping, in which you have an idea as to what you want people to hear and you find information that supports that idea. Walker has argued that the monthly numbers are based on a small sample of the state (3.5 percent of employers), that the numbers are volatile and that the margin of error is so large, it almost isn’t worth looking at the data’s outcomes. Experts in the field support and refute various aspects of those things to some degree, but what concerns me most is the lack of full analysis done with these numbers.

Journalists can tend to parrot sources when they feel a lack of confidence in their ability to understand what something is actually revealing. In addition, the concept of “fair and balanced” has degraded to the point of throwing a new fact into the arena and letting both sides attack it with soundbites. Don’t worry that the readers won’t understand this stuff. They probably aren’t reading your paper anyway.

Here are several key components of what go untouched:

– The City Mouse Effect: Milwaukee is both the largest city in the state and the home base for both of these idiots. Job losses are more likely to be concentrated in that area than in other places, especially in an economic downturn. Sure, a paper plant in Niagra closing will devastate the community and put about 50-60 percent of the town out of work. However, it’s a small town and the losses, when measured in pure numbers, won’t be as great as if a paper plant closes in the Fox Valley or if Ladish or Caterpillar closes in Milwaukee. In addition, when a downturn kills or severely harms an industry, that impact will be more likely to be felt in a city like Milwaukee because it will host multiple places that service that industry. For example, if an economic downturn leads to unemployment for foundry workers, a small town is likely to have a single foundry. This will lead to cuts at that one factory. If the downturn impacts a city like Milwaukee, they have multiple foundries in which the impact will be felt. And that says nothing about the collateral damage associated with a single job loss.

– Covariance: In assessing any set of data, the first question you should ask is if any other variable might be impacting the variable you are trying to measure. For example, let’s say you have the idea that college students who watch TV news will have a greater fear of being sexually assaulted on their campus.

Of course, you need to measure fear of sexual assault on campus and television viewing habits, but that’s not enough. Things like the gender of the participant, the size of the campus, the location of the campus (urban/suburban) and a half dozen other factors should be examined. Once you parse out these covariates, you can see what’s left and whether a direct line can be drawn between these two variables.

One of the big “other” factors that has me scratching my head is the Milwaukee job numbers. Walker’s ads paint Milwaukee as a shithole where jobs go to die while simultaneously touting huge job increases throughout the state.

What hasn’t happened (on either side of this campaign) is a decent analysis of the state as a whole not counting Milwaukee. In other words, extract the covariate that is Milwaukee (call it a draw between Barrett and Walker, as neither could really make a compelling argument that it was he who brought a specific job to Milwaukee) and then analyze the data.

Just for the sake of argument, here are the unemployment numbers and percentages (you need to rerun the search if you want to see the numbers) for Milwaukee in 2012:

January: 60,909 (7.7)
February: 63,738 (8.0)
March: 61,276 (7.7)

And here are the state’s:
January: 229,403 (7.6)
February: 240,995 (7.9)
March: 229,801 (7.5)

If you redact Milwaukee, here’s what you get:
January: 168,494 (7.5)
February: 177,257 (7.9)
March: 168,525 (7.5)

In other words, the alleged shithole Tom Barrett has been running for a decadeis in no way different than the remainder of the state in terms of percentages. In fact, this could actually be a win for Barrett if he simply said, “Look, we’re responsible for more than a quarter of this state’s workforce, which means when the economy takes a shit, we are most vulnerable to data spikes. Instead, we’re right with the rest of the state. Put that in your pipe and smoke it.”

– Attribution: When data are reported like those in the set Walker released, each job is counted as a data point. However, what is missing here is where those jobs are placed and to what can we attribute their placement. In other words, if a Democratic mayor in a small town convinced a friend to start a small business and hire five people, those are six data points that Walker can count that really have nothing to do with him or his policies. The same is true if someone moves their fireworks business from one side of the Wisconsin/Illinois border to the other in order to take advantage of a sales tax disparity.
What the surveys done at the monthly and quarterly level do is simply measure quantitative movement, not why that movement occurred. In addition, any job that Barrett might have created will automatically count as a new state job as well, leading to the “rising tide” outcome of improved job conditions in the state. Where these jobs were created and WHY they were created becomes critically important in analyzing this data. However, this hasn’t stopped Walker from telling everyone in every possible ad that “Our reforms are working!”

By not really looking at this data in any kind of meaningful way, the Democrats are allowing the Republicans to play this issue to a draw. The Democrats appear to be receding from the issue as we speak, hammering more now on the ongoing John Doe probe instead of taking the time to really crunch the numbers. With the numbers continuing to change and Barrett’s failure to attack the “Milwaukee is a job killer” ads, the narrative is allowed to emerge that the state’s largest city is a millstone around the neck of Wisconsin.

The truth is a lot more nuanced. The city is in no way different from a statistical point of view than the state when you look at the data. In addition, no data has emerged that indicates any job gains have been the result of a specific trigger, such as Walker’s policies or actions.

The saddest thing is that it’s unclear if making this known would be worth the trouble. Recent polling data seem to indicate that most people have made up their minds already about this election.

And that’s scary as hell.

4 thoughts on “Confounding Variables

  1. And let’s not forget the role that Milwaukee County played in all of this. What were the rates of unemployment in the County during the same time period when it was being run by who? Oh, yeah, Walker. There’s also an argument that can be made that the county has an outsized influence on economic development in the city, which if it correlates would mean Walker is as much or more responsible for the City of Milwaukee’s unemployment problem.

  2. “And how does WI compare to the rest of the USA?”
    Answering that is one of the problems with the number Walker is using. It might very well turn out that the Walker number is the more accurate, but we won’t know that for at least another six weeks (or something like that). And even if it is, we won’t know how it compares to other states for at least that long.
    Part of the problem for Walker is that the more quickly available ‘survey’ number has him/WI in last place in the USA in job creation. Literally, 50th out of 50. Even if the ‘census’ number moves things from negative to positive – as you ask – ho does it compare? We won’t know until after the recall vote.

  3. I overheard a new turn of speech today. A teacher was talking to someone I didn’t recognize about a third person. The professor said, “Her last day is May 31st. She’s been Walkered.”
    Just so you know.

  4. And how does WI compare to the rest of the USA?
    Attributed to Mark Twain – There are lies, damn lies, and statistics.
    One book I recommend a lot for the math phobic is the small book “How to Lie with Statistics” (along with its sequel). It is getting kind of old, but it gets folks thinking the right way of how to look out for these things. The basic premise of the book is that the crooks already know how to break into your home, so you need to know the same info in order to protect yourself. Treatment uses almost no math whatsoever.

Comments are closed.