A Huge Victory for Science, Too

UPDATE: OK, this was so blindingly obvious that half the bloggers and pundits in the world came up with it. Can’t win ‘em all.

In the end, my favorite take on it has to be Rachel Maddow’s.


When I say that last night’s events are a huge win for science, I don’t mean that the funding situation is more promising, though of course the result is far more benign for the scientific community than the alternative “tea-party” beholden administration would have been.

I refer rather to the stunning demonstration of the value of quantitative expertise that has been provided by Nate Silver and his critics, and the very impressive demonstration of how people without quantitative skills would be wise to defer to those who have them. This may end up being a consciousness-changing event in the Beltway culture, and it is very much to the advantage not so much of science as an institution nor of science as a body of knowledge, but to the renewed respect for science as a process within the capital culture.

I’m not the only person to notice the similarity between climate denialism and “Nate Silver denialism”. To someone interested in both climate and US politics, it is plain as day. Various tweets confirm. Even Sarcastic Rover gets in on the act. (Lots more where that came from, as a little Twitter searching can easily reveal.) But even though it verges on a ploddingly obvious connection, I can’t resist adding my two cents, since this all was very revealing on some of the key topics that exercise us here. Specifically, why does the press spin climate science and sustainability matters so execrably, and what can we do about it.

Let me offer some background for the apolitical and those watching from a decent distance in other countries.

Nate Silver is a data nerd.

American elections are a delightful field of endeavor for the data nerd, surely unequaled in other countries’ electoral processes. This is attributable to the antiquated and ill-adapted method of voting, wherein the majority in a state, however slim, votes in proportion to the population of their state. (It’s also slightly skewed to the smaller states but that is neither here nor there.) Thus, a presidential election effectively constitutes 50 simultaneous mini-elections with effectively identical choices, among strongly overlapping but hardly identical demographics. Economies of scale appear in the process of public opinion polling. Motivated by the political sector itself, a few organizations conduct such polling.

Because America is a two-party system the dynamics of democracy pull strongly for a very close split. One aspect that is remarkable is how thoroughly polarized these sides have become, but that doesn’t concern the thrust of my argument here. Rather, the point is that we a handful of organizations produce a handful of polls gauging the opinion of 50 otherwise identical elections with a variety of overlapping demographics. Now these polls are noisy. We have been hearing in various states and nationally, it’s Obama, it’s Romney, back and forth. Huge spikes appear even in the individual calculations of a particular state by a particular polling agency. Margins of error are typically quoted as +/- 4% or 5%, and margins of victory in key states are substantially lower than the uncertainty.

That one could obtain a viable prediction from such data is quite a reasonable expectation from those who are familiar with quantitative thinking, but to others who are not, it seems absurd. If you have twenty polls with a five per cent margin of error and an eight percent range, surely combining them makes for a huge 18% uncertainty, which is to say your guess is as good as mine. More to the point, from the professional commentator’s point of view, my guess is at least as good as yours.

The idea that more information is hiding in the copious data than might be casually obvious is anathema to some of them. They didn’t pick political journalism as a career because they enjoy math.

 

Their defense was simple. Nate Silver, they argue, called 49 out of 50 states correctly in 2008 out of sheer luck. Lots of people made predictions, I suppose, so one of them might just get lucky. There is no reason to make a New York Times blogger out of him for that, no reason to believe that his predictions will hold yet again. Besides, it is more dramatic if Romney has the momentum, which he obviously has, see?

We’re familiar with this at the most rudimentary and empirical levels in climate. I don’t have a good temperature record here and I don’t have a good one there, I hardly have one anywhere. So why am I so sure about the damned hockey stick?

In both the political and climate prognoses, the technical answer is broadly the same. Uncertainties do not add. More information reduces uncertainty. That’s why it’s called information. The set of all polls contains more information than the polls from any individual organization.

C. P. Snow complained a half century ago about the “two cultures” problem in academia; the lack of acculturation among scientists and the lack of rational constraint among liberal arts types. And while it’s true that there’s a fair amount of autism spectrum personality types among scientists (though not as much as among engineers, I’d wager) it turns out to be easier to add culture to a scientific worldview than to add science to a culture-focused one. Some folk just really hate math, and that their bread and butter job can be better done by an algorithm terrifies them. So denial is unsurprising. As Al Gore pointed out in his movie, it’s hard to convince someone of a fact when that person’s livelihood depends on disbelieving it.

Reporters on the science beat seem to feel very much like reporters on the political beat, that it is their job to weigh the evidence and present “narratives” that pass their gut check. The Nate Silver incident gives us another instance where we can watch them fail to identify where the real expertise is, and do so out of loyalty to their own professions rather than to the truth. And besides, looking too carefully just gives them a headache.

Anyway, the point is this is not going away. It really may be a turning point for the injection of objective methods into public discourse, which obviously have been at an appalling low point. How? Well, what about Silver’s final predictions for the 2012 presidential election? Here they are:

He was on the right side of the line on at least 49 out of 50. (Florida was still undecided last I looked.) 50 out of 50. Even Republicans noticed this.


images: Slate, xkcd, Nate Silver’s blog.



UPDATE: Joe Romm makes the connection too, (h/t R L Miller) pointing to this peculiar tweet from David Frum:

but Frum should be cut some slack here. He was being sarcastic.


UPDATE: Jay Rosen points to this remarkable example of strident, willful cluelessness among the old school pundits:

Time magazine’s Joe Klein told POLITICO. “Polling is inexact, especially with the cell phone factor — not enough data over time for pollsters to be absolutely sure they’re getting it right.”

In a column headlined “I don’t know,” Klein wrote earlier this week, “Anyone who claims to know who is going to win is blowing smoke.”


UPDATE: In comments, Dan Olner points to Krugman’s column The War on Objectivity, defending Silver before the outcome was known. Krugman may have been the first to publicly compare Nate Silver denialism with climate denialism among others:

They know, just know, that Nate must be cooking the books. How do they know this? Well, his results look good for Obama, so it must be a cheat. Never mind the fact that Nate tells us all exactly how he does it, and that he hasn’t changed the formula at all.

This is, of course, reminiscent of the attack on the Bureau of Labor Statistics — not to mention the attacks on climate science and much more. On the right, apparently, there is no such thing as an objective calculation. Everything must have a political motive.

This is really scary. It means that if these people triumph, science — or any kind of scholarship — will become impossible. Everything must pass a political test; if it isn’t what the right wants to hear, the messenger is subjected to a smear campaign.


UPDATE: Rachel Maddow makes the point in free verse


UPDATE: Gavin beats me to the punch!


UPDATE: Peter Gleick tweets: the success of @fivethirtyeight is not what makes #climate science right. It would be right even if his projections had been off.

Comments:

    • Good idea, thanks. Done. Also fixed what is probably a record number of typos and spelling errors, even for me. (Posted late last night.)

  1. My initial reaction was to suggest peddling some simple examples of where sampling uncertainty comes from, but then I remembered how difficult it seems to be to introduce some people to where the normal distribution comes from.

    Also, in Krugman-reposting-autobot mode:

    Yet the right — and we’re not talking about the fringe here, we’re talking about mainstream commentators and publications — has been screaming “bias”! They know, just know, that Nate must be cooking the books. How do they know this? Well, his results look good for Obama, so it must be a cheat. Never mind the fact that Nate tells us all exactly how he does it, and that he hasn’t changed the formula at all...

    This is really scary. It means that if these people triumph, science — or any kind of scholarship — will become impossible. Everything must pass a political test; if it isn’t what the right wants to hear, the messenger is subjected to a smear campaign.

    So yes - a lucky escape.

  2. Jason Zengerle at New York Magazine argues that this Silver lining has a cloud:

    "Without a doubt, Silver’s rigorous empiricism is much, much more preferable to the lazy, gassy, vibration-sensing punditry that has made up so much of our political journalism. And yet, the biggest complaint about campaign coverage over the last twenty years has been that it’s too focused on the horse race and doesn’t pay enough attention to the substance. Silver and his fellow polling analysts and aggregators have brought a welcome degree of precision, but they’ve only made the horse race more central to the political conversation. After all, what dominated that conversation for the past month? It wasn’t a conversation about the candidate’s dueling tax plans. Rather, it was a debate about the polls. The fact that the good guys — who put their faith in the data rather than the vibrations — won that debate may turn out to be something of a pyrrhic victory"

    To the contrary, I have seen it argued (I don't recall where) that moving the horse race out of reach of punditry may cause them to focus on the actual issues that actually make the election a matter of some actual importance.

  3. Just the teeniest bit OT, but maybe not really: MT refers to the US Electoral College as "slightly skewed to the smaller states". True. Something too often taken for granted is that the US Senate, not coincidentally, is grossly skewed to the smaller states, which, in practice, means the less urban and far more rightward states. This started out as a messy compromise in the 18th century, and has grown to something closer to a crime against humanity.

    This arrangement is indefensible. (For someone who has spent as much time around the academy as I have, that's way worse than wrong.) Think for a few moments about recent years when the Dems have had about 59 of 100 votes in the Senate, representing 65+ % of the population, and some of the legislation they have struggled to pass. A climate bill frinstance. How different the voting, and the bill itself, might have been if a resident of Wyoming had the same say as one from California, instead of dozens of times the weight.

  4. I would be a whole lot happier if Nate Silver hadn't presumed to know more than he does, and listen to parties that were bent on deception, in the chapter on climate change. This fail will become more prominent with Silver's new popularity. It is a serious problem. While in general I am not a fan of extended extracts, on re-rereading, I think almost all of what Mike Mann had to say was relevant, so am not excising. There's a bit more at the link.

    http://www.huffingtonpost.com/michael-e-mann/nate-silver-climate-change_b_1909482.html

    It's not that Nate revealed himself to be a climate change denier; He accepts that human-caused climate change is real, and that it represents a challenge and potential threat. But he falls victim to a fallacy that has become all too common among those who view the issue through the prism of economics rather than science. Nate conflates problems of prediction in the realm of human behavior -- where there are no fundamental governing 'laws' and any "predictions" are potentially laden with subjective and untestable assumptions -- with problems such as climate change, which are governed by laws of physics, like the greenhouse effect, that are true whether or not you choose to believe them.

    Nate devotes far too much space to the highly questionable claims of a University of Pennsylvania marketing Professor named J. Scott Armstrong. Armstrong made a name for himself in denialist circles back in 2007 by denouncing climate models as having no predictive value at all. Armstrong's arguments were fundamentally flawed, belied by a large body of primary scientific literature -- with which Armstrong was apparently unfamiliar -- demonstrating that climate model projections clearly do in fact out-perform naive predictions which ignore the effect of increasing greenhouse gas concentrations. As discussed in detail by my RealClimate.org co-founder, NASA scientist Gavin Schmidt, Armstrong simply didn't understand the science well enough to properly interpret, let alone, assess, the predictive skill of climate model predictions.

    That Nate would parrot Armstrong's flawed arguments is a major disappointment, especially because there are some obvious red flags that even the most cursory research should have turned up. A simple check of either SourceWatch or fossil fuel industry watchdog ExxonSecrets, reveals that Armstrong is a well-known climate change denier with close ties to fossil fuel industry front groups like the Heartland Institute, which earlier this year campaigned to compare people who accept the reality of climate change to the Unabomber, and secretly planned to infiltrate elementary schools across the country with industry-funded climate change denial propaganda. I suspect that Nate's failing here arises from a sort of cultural bias. There is a whole community of pundits with origins in economics and marketing who seem more than happy to dismiss the laws of physics when they conflict with their philosophy of an unregulated market. Nate may not share that philosophy, but he was educated by those who do.

    Nate Silver was trained in the Chicago school of Economics, famously characterized by its philosophy of free market fundamentalism. In addition to courses from Milton Friedman, Nate might very well have taken a course from University of Chicago economist Steven Levitt, known largely for his provocative 2005 book Freakonomics and its even more audacious 2009 sequel Super Freakonomics -- a book that, perhaps better than any other, serves as a cautionary tale of the dangers that lurk when academics attempt to draw sweeping conclusions in fields well outside their area of training. In Super Freakonomics as you might guess, Levitt drew questionable conclusions about climate change and related energy issues based on an extrapolation of principles of economics way, way, way, outside their domain of applicability. Even some very basic physics calculations, for example, reveal that his dismissal of solar energy as a viable alternative to fossil fuel energy in combating climate change because of possible waste heat is total nonsense. Ray Pierrehumbert, a chaired professor himself at the University of Chicago, in the Department of Geosciences, pointed this and other serious errors out to Levitt in an open letter that concluded with a campus map showing how easy it would have been for Levitt to walk over to his office to discuss his ideas and, presumably, avoid the serious pitfalls that ended up undermining much of what he ended up saying in his book about climate change and energy policy.

    Unlike Levitt, Nate did talk to the scientists (I know. I'm one of them!). But he didn't listen quite as carefully as he should have. When it came to areas like climate change well outside his own expertise, he to some extent fell into the same "one trick pony" trap that was the downfall of Levitt (and arguably others like Malcolm Gladwell in The Tipping Point). That is, he repeatedly invokes the alluring, but fundamentally unsound, principle that simple ideas about forecasting and prediction from one field, like economics, can readily be appropriated and applied to completely different fields, without a solid grounding in the principles, assumptions, and methods of those fields. It just doesn't work that way (though Nate, to his credit, does at least allude to that in his discussion of Armstrong's evaluation of climate forecasts).

    As a result, Nate's chapter on climate change (Chapter 12: "A Climate of Healthy Skepticism") is marred by straw man claims that don't stand up to scrutiny. These include the assertion that (a) climate scientist James Hansen's famous 1988 predictions overestimated global warming (they didn't), that (b) "the Intergovernmental Panel on Climate Change (IPCC) settles on just one forecast that is endorsed by the entire group" (pure nonsense -- even the most casual reading of the IPCC reports reveals that great care taken to emphasize the non-trivial spread among model predictions, and to denote regions where there is substantial disagreement between the projections from different models) and that (c) "relatively little is understood" about the El Nino cycle (here I imagine that Nate might have misinterpreted our own discussion about the matter; I explained in our discussion that there are still open questions about how climate change will influence the El Nino phenomenon -- but that hardly means that we know "relatively little" about the phenomenon itself! In fact, we know quite a bit about it). Finally, and perhaps most troubling (d) while Nate's chapter title explicitly acknowledges the importance of distinguishing "signal" from "noise", and Nate does gives this topic some lip service, he repeatedly falls victim to the fallacy that tracking year-to-year fluctuations in temperature (the noise) can tell us something about predictions of global warming trends (the signal). They can't -- they really can't.

    Nate's view of uncertainty, and its implications for climate model predictions, is particularly misguided. He asserts that the projections of the IPCC forecasts have been "too aggressive", but that is simply wrong. It neglects that in many cases, e.g. as regards the alarming rate of Arctic sea ice decline (we saw a new record low set just weeks ago), the climate models have been far too cautious; We are decades ahead of schedule relative to what the models predicted. Uncertainty cuts both ways, and in many respects -- be it the rapid decline in Arctic sea ice, or the melting of the ice sheets -- it is cutting against us. Uncertainty, as many economists recognize, is thus a reason for action, not inaction! I'm surprised someone as sharp as Nate just doesn't appear to get that.

    Nate also takes some unnecessary cheap shots. In what has now become a rite of passage for those looking to establish their "honest broker" bona fides in the climate change debate, Nate makes the requisite "punch the hippie" accusation that Al Gore exaggerated the science of climate change in An Inconvenient Truth (a team of climate scientists reviewed the movie for accuracy and found that by-and-large Gore got the science right). He characterizes climate scientist Gavin Schmidt as a "sarcastic" individual who is unwilling to put his money where his mouth is by betting his personal savings on his climate model predictions (this felt to me reminiscent of Mitt Romney's widely mocked $10,000 bet challenge to Rick Perry). And while I do appreciate some of the nice things Nate says in the book about me personally (e.g. "Mann is exceptionally thoughtful about the science behind global warming"), he at the same time deeply misrepresents our discussion on several counts.

    I had emphasized the importance of distinguishing the true uncertainties in climate science (and there are plenty e.g. the influence of warming on hurricanes, how the El Nino phenomenon might be affected, or how regional patterns of rainfall may change) from the manufactured uncertainties and myths typically promoted by climate change deniers and contrarians (e.g. "how come there has been no warming since 1998?" -- the answer is that, of course, there has been). I stressed how important it is, when scientists communicate to the public, to make clear that while there are many details that are still uncertain, the big picture (that humans are warming the planet and changing the climate, and that far larger and potentially more dangerous changes loom in our future if we don't act) is not.

    Nate cherry-picks a single sound bite ("our statements [should not be] so laden in uncertainty that no one even listens.") to once again reinforce the false narrative that scientists are understating uncertainty. The point I was actually making was that we cannot spend so much time talking about what we don't know, that we don't end up telling the public what we do know. That, as Nate correctly quotes me, "would be irresponsible". Nate states that "the more dramatic [climate scientists'] claims, the more likely they [are] be quoted...", seemingly implying that scientists have a motivation to overstate the science. He ignores the fact that those scientists willing to feed the false "scientists are exaggerating" narrative are the true darlings of the "balance" over "objectivity" school of news reporting -- a school of thought that Nate sadly seems to have subscribed to.

    Most disappointing to me of all was the false equivalence that Nate draws between the scientific community's efforts to fight back against intentional distortions and attacks by an industry-funded attack machine, and the efforts of that attack machine itself. He characterizes this simply as a battle between "consensus" scientists and "skeptical" individuals, as if we're talking about two worthy adversaries in a battle. This framing is flawed on multiple levels, not the least of which is that those he calls "skeptics" are in fact typically no such thing. There is a difference between honest skepticism -- something that is not only valuable but necessary for the progress of science -- and pseudo-skepticism, i.e. denialism posing as "skepticism" for the sake of obscuring, rather than clarifying, what is known.

    Nate deeply mischaracterizes an editorial published by the prestigious and staid journal Nature (whose sentiments are echoed in my book The Hockey Stick and the Climate Wars) warning scientists that they "must acknowledge that they are in a street fight, and that their relationship with the media really matters." Nate grossly mischaracterizes the quote, claiming that "the long-term goal of the street fight is to persuade the public and policy makers about the urgency (or lack thereof) of action to combat climate change." Nate makes it sound like the "street fight" was of the scientists choosing, completely turning on its head what Nature was actually talking about: scientists finding a better way to defend science from cynical attacks whose sole aim is to confuse the public about what we actually do know about climate change (and therefore forestall any efforts to deal with it).

    I could detail the numerous other problems with the chapter (and no -- there aren't really 538 of them; I confess to having taken some "poetic license" with the title of this commentary). But the real point is that this book was a lost opportunity when it comes to the topic of climate change. Nate could have applied his considerable acumen and insight to shed light on this important topic. But the result was instead a very mixed bag of otherwise useful commentary marred by needless misconceptions and inappropriately laundered denialist memes.

    Don't get me wrong. I'm still a FON (Fan Of Nate). I will continue to follow his thoughtful commentary on all matters of politics and polling. But when he makes claims about other topics, like climate change, I think I'll be a lot more skeptical. Skepticism -- real skepticism -- is, after all -- a good thing.

  5. Experience with forecasting noisy Boston weather shows that consensus outperforms any individual over time, even when the composition of the consensus ranges from freshmen through grad students and professors with 40+ years of experience:
    ftp://texmex.mit.edu/pub/emanuel/Sanders/Sanders_1986_Trends.pdf


Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>