The Fifty States of Nate

Dear readers,

Today’s post deals with psephology, the study of elections, and more specifically, the application of mathematics and statistics to it. This field has gained notoriety over the past few years in the United States, largely thanks to the spectacularly successful (and often attacked) predictions by Nate Silver in his blog. While I am not a psephology expert by any means, I understand enough about Nate Silver’s models, and mathematical models in general, to hopefully provide you with an interesting perspective.

The first point I want to make is that, although Nate Silver did predict the outcome of the 2012 United States elections correctly in all 50 states, this in itself is neither his most significant achievement nor an extremely impressive one. His most significant achievement is to highlight the principled application of mathematics and statistics to an area that has up to now been subject to large amounts of human bias. In this, he is far from being alone; however, thanks to his blog’s association with the New York Times, he is the most visible. I give him credit for drawing attention to the power of mathematics and statistics.

Nate Silver is also far from providing the most accurate predictions of the outcome of the last elections. In a detailed analysis, the Center for Applied Rationality actually shows that predictions made by Drew Linzer and Sam Wang were somewhat more accurate when not only the prediction, but also the fraction voting for each party, are taken into account. As for whether predicting 50 outcomes correctly is in fact impressive, consider that there was considerable uncertainty in only 9 of the 50 states. This means that even a completely random prediction made by 9 coin flips had a roughly 0.2% chance of being perfect. Of course, this probability goes up tremendously if good data is available from election polls.

Furthermore, while mathematical models are great for bringing together data from different sources (in Nate’s case, poll data, as well as some understanding of how voter preferences change over time), they are also vulnerable to the biases of this input data. In particular, if the data sources themselves are noisy or inaccurate, even a good mathematical model may go astray. This phenomenon, known as garbage in, garbage out, was partly responsible for Nate Silver’s rather poor prediction of the outcome of the 2010 UK elections. Indeed, poll data in the UK is not disaggregated by region as it is in the US, so that more uncertainty about the regional variation was necessarily present which appears to have thrown Nate off.

About a month before the 2012 US election, Nate Silver himself pointed out, “I’m sure that I have a lot riding on the outcome […]. I’m also sure I’ll get too much credit if the prediction is right and too much blame if it is wrong.” This brings me to my next point. Although mathematical models are frequently used to predict reality, this is neither their only nor even their main use. It may seem like a good idea to evaluate models by how well they predict relaity, but their real value is in helping us understand which factors (or variables) are important and which are less important. Their main role is to provide insights into complex phenomena, whether they are climate change, etiology of diseases, or election outcomes.

Unfortunately, Nate Silver seems to miss the opportunity to allow his models to provide these insights by keeping them private. This is the main concern I have about his work. Perhaps the academic idealist in me wants to be able to reproduce the results of any modeling effort, if only in principle, and Nate may have valid commercial or ethical reasons for not disclosing his models. Still, it would be a good idea for the sake of transparency to allow his fellow psephologists to look “under the hood”, just like the aforementioned Drew Linzer, Sam Wang, and many others do. I strongly believe that transparency and open sharing is critical for advancing the field, as well as preventing the reliance on “tweaks” that may temporarily improve model performance, but are detrimental in the long run.

In any case, I just added Nate Silver’s recent book to my reading list. It seems that the book tackles a wide variety of applications of mathematical modeling (much like I’m planning to do in this blog). I’ll be sure to read it over the New Year break and report on the experience, and I welcome your comments if you’ve already had a chance to read it, or simply want to comment on another issue discussed here.

Is a Museum of Mathematics necessary?

Dear readers,

Today, 12/12/12, is an important date for mathematics in the USA. Not just because it is a date which has three identical components (something that we won’t see again for another 88-odd years), and not even because it’s one of the 12 dates every year that are abbreviated the same way in the USA and the rest of the world (where, as you likely know, they are written with the days before the month). It is also significant because it marks the opening ceremony of the Museum of Mathematics in New York City.

If you read the New York Times, you may recognize the title of this post as a play on an article that appeared there this past summer, entitled Is Algebra Necessary? This misguided and naïve article was among the motivating factors for this blog, as it made me realize how undervalued and misunderstood mathematics education is even among the educated. Several good rebuttals to this article failed to get a larger conversation started, and it is this larger conversation that I want to contribute to with this blog.

Mathematics education is a topic I plan to address from many different angles next year, but for today, I will simply discuss whether a Museum of Mathematics is a step in the right direction. Glen Whitney, the museum’s director, told the New York Times that its mission is to shape cultural attitudes and dispel the bad rap that most people give math – which happens to be one of my blog’s objectives as well, so I could not agree more with his motivation. The big question is whether a museum can achieve this goal.

My main concern is that an appreciation for mathematics is unlikely to come from visiting exhibits in a museum – rather, I believe it can only appear from positive exposure to mathematics in everyday life. If a child dislikes the subject in school, they may expect going to a math museum to be boring. Just as with art, music or a language, a taste for mathematics is an acquired one. It usually requires a certain level of maturity and discipline, which children are unlikely to acquire without outside encouragement.

For this reason, parental influence, as well as that of peers, is critical. An environment that views math in a positive light will have its effect with or without the museum, while a hostile environment may not prepare a child for being receptive towards the museum’s exhibits. Without innovative changes in the mathematics curriculum, a more favorable attitude towards the subject, and a deeper understanding of the role of mathematics in our culture, the impact of the Museum of Mathematics may remain limited.

Despite my skepticism I remain open-minded. On my upcoming trip to New York City I will make sure to stop by the museum and see how deeply its exhibits engage me. I will report on my experience in an upcoming blog post, so stay tuned. Perhaps abstract concepts really can be made fascinating – and concrete – in a museum setting. I am just not sure that the museum can make enough of a difference.

There is, however, at least one thing that I’m pretty sure about. Dr. Whitney spent a decade working at an investment firm, and, in his own words, decided to engage in an activity with “more direct socially redeeming value”. While the social impact of his current undertaking may be limited, it will at least be significantly more positive than that of his previous job. And that, in itself, is definitely a good thing!

Mathematicians say the darndest things!

Dear readers,

If you have mathematically inclined friends or are mathematically inclined yourself, you may know that mathematicians tend to use a very specialized language when discussing their work. This is not a matter of trying to make the subject of discussion appear inaccessible to outsiders (though sometimes there is an element of that). Instead, it’s a communication necessity: everyday language is even more inadequate for discussing mathematics than it is for discussing topics such as music, philosophy or law.

But the terms used in mathematics are rarely invented from scratch; a great number of them are in fact taken from everyday language, but given different meanings. This is why you might hear words like filter (in lattice theory), flag (in linear algebra), ring (in abstract algebra), saddle (in calculus and analysis), sheaf (in topology), soul (in geometry), at a mathematics lecture. Their meanings, though usually inspired by natural language, frequently drift to the point of becoming unrecognizable.

Fortunately, this exchange between mathematics and natural language is a two-way street, and today I’m going to discuss some of the words that mathematicians often use in day-to-day conversations that are derived from mathematical terms. Using them might make you sound strange, but it might also help you express things concisely. I present to you the top 5 words I’ve heard used in conversations between mathematicians (and have been guilty of using myself in conversations with non-mathematicians on occasion). Whether you decide to adopt them or not, you’ll find it illuminating to know their meanings.

Mathematical meaning: as the independent variable becomes arbitrarily large.
Colloquial meaning: in the long run; looking at the big picture.
Example 1: Since he’s not staying in academia, having his name on papers is asymptotically irrelevant.
Example 2: They do have a significant age difference, but asymptotically it won’t matter all that much.

Mathematical meaning: an easily derived logical consequence of a proven result or theorem.
Colloquial meaning: something that follows automatically; something you get for free.
Example 1: If she gets invited to this wedding, her significant other’s presence will be a corollary.
Example 2: I just bought myself a new bicycle, and this water bottle came as a corollary.

Inflection point
Mathematical meaning: the point on a curve at which the second derivative changes sign.
Colloquial meaning: a situation where each new unit of effort affects the result less than the one before.
Example 1: I could have stayed up all night to finish it, but I don’t like to work past my inflection point.
Example 2: We’ve been working for ten hours and the inflection point is getting close; let’s call it a day.

Mathematical meaning: the remainder of the division of a given quantity by another (fixed) quantity.
Colloquial meaning: minor details that must be dealt with as a condition for getting a desired outcome.
Example 1: I will definitely join you at the basketball game tonight modulo finishing this assignment.
Example 2: I got the job modulo the background check and some annoying paperwork.

Mathematical meaning: perpendicular; in probability theory, uncorrelated (refers to random variables).
Colloquial meaning: unrelated; independent.
Example 1: Your observation is very interesting, but it’s orthogonal to the question we are discussing.
Example 2: How important a task is is often orthogonal to how urgent it appears to be.

Are there any other mathematical terms you like to use in everyday life? Share them in the comments!

My first date and stereotypes about mathematicians

Dear readers,

My first date (at the end of high school) involved going to see “A Beautiful Mind” with a girl I liked. At some point during the movie, she told me: “Leonid, please don’t become like this”. While this was a very thoughtful comment, it made me realize that even intelligent and well-educated people may believe that mathematicians resemble John Nash as he is represented in the movie. Another widespread image of a mathematician is that of Grigori Perelman, of the Poincare conjecture fame. This is why this blog post discusses some stereotypes about mathematicians – and their work – prevalent in our society.

Of course, almost all stereotypes (except for the very outlandish ones) are based in reality. The problem with stereotypes is not that they are false, but that they are too general, and substitute a generalization that frequently fails to apply for our ability to judge people for who they are. Stereotypes provide significant savings of mental and emotional effort, which is why they persist, but they also come with a high opportunity cost – that of dismissing a chance to relate to an individual. One goal of my blog is to prevent you from paying this cost when it comes to mathematicians; another goal is to show that they are generally interesting people worth getting to know, which I plan to do through my interview series.

So first, let’s close our eyes and imagine a mathematician based on what we know from popular culture and the media, conjuring up as precise an image as we can, including physical appearance as well as character traits. Chances are, the image you have will look something like this: a white male in his late thirties, with clothes covered in chalk dust and hair in disarray, self-absorbed but with eyes lighting up when working on a challenging mathematical problem. This might be someone who enjoys being alone more than interacting with other people, possesses limited social skills and lives in his own little world.

Now, let’s examine each of these characteristics in turn to see whether they correspond to reality. Rather than rely on data, which is rather sparse, I will draw on my personal experience to give you a more nuanced picture. Along the way we will discover some of the real quirks of real mathematicians and see which characteristics are really helpful to a successful mathematical career and which are a distraction.

We often hear that “mathematics is a young man’s game”, a statement I used to believe, which made me sad to turn 21, an age at which Gauss had a PhD and Galois, eponym of the famous theory, was already dead. In reality, mathematicians mature at different rates, and many achieve their best results after 40 – Andrew Wiles’ proof of Fermat’s Last Theorem, which had remained unproven for over 350 years, is a case in point. On the other hand, mathematics research gets harder with age, though many, like Harold Coxeter, last century’s greatest geometer, continue doing it into their 90s. One of my mentors who has recently turned 67 once told me that he’d like to live forever as there is always more mathematics to do.

Although our statement probably uses “man” generically, we need to acknowledge that women are still much less likely to do mathematics than men professionally. I’m convinced that the reasons for this are social rather than genetic, despite Larry Summers’ infamous claims to the contrary. In fact, several of my best mentors, including my PhD advisor, were women. Yet the question of how best to encourage young women to develop their mathematical ability is challenging. A campaign like Science: It’s a Girl Thing strikes me as the wrong way to approach this problem. On the other hand, Danica McKellar’s books, which encourage girls to excel in high school mathematics, made a favorable impression on me.

I’m equally convinced that social rather than genetic reasons are responsible for Caucasians currently being over-represented among mathematicians. Mathematics traces its origins to Africa, owes a great many discoveries to Indian and Arab mathematicians during the Middle Ages, benefits from important contributions coming out of China, and only became predominantly European in the last few centuries. Even Andrew Wiles’ breakthrough built on the work of the Japanese mathematicians Iwasawa, Shimura and Taniyama, to give a modern example. African American, Chinese, Indian, Korean, Latin American, Persian and Vietnamese students were integral to my MIT Math PhD program and greatly enriched it.

When it comes to sartorial choices, mathematicians I know do have a propensity for prefering informal clothes, because they value function over form. This year I attended several professional conferences, including the largest annual mathematics convention held in Boston in January. Sure, it was no business meeting or fashion show, but none of its seven thousand attendees stood out for their lack of attention to self-presentation. As for chalk dust, it can be seen on those mathematicians who teach old style, and chalk allergy is an occupational hazard of mathematics if there ever was one. Then again, most people from my generation use slideshow presentations to teach and whiteboards with markers to do research.

As for personality, doing mathematics requires an ability to reason about abstract concepts for extended periods of time. This demands commitment and hard work, not unlike making art or playing music. No wonder that the annual Math Department Music Recital at MIT is never short of performers, or that my best friend from the Math PhD program is also a talented artist. Most mathematicians I know, including myself, are indeed introverted and prefer to work alone. At the same time, the majority of mathematical papers coming out today have multiple authors; none of my 8 papers published so far was written solo. So, at the end of the day, mathematics, just like music or art, is meant to be shared with the community.

Now, in many of our minds, being introverted is frequently associated with having limited social skills. I prefer to think of it as a low tolerance for small talk and a limited interest in other people unless they are particularly engaging. For many introverts, myself included, developing these skills is only possible through a concerted effort, and there are many among us who never make the decision to invest time in it. This may make it harder to get to know mathematicians on a personal level, especially for extroverts. However, the rewards of trying to do so may be significant; mathematicians’ opinions about everything from the history of our civilization to human reproductive biology will be sure to challenge your views.

As for living in their own world, the specialized language mathematicians use (the topic of my next post) might give this impression. But most mathematicians I know do actively engage with the world around them, and only a minority tends to dismiss it as a distraction to their work, like the quirky Paul Erdos. Thus, one of my fellow PhD students at MIT has a side career in ballroom dancing, another is a competitive programmer, and still another is contemplating a career in politics. I’ve been very actively engaged in social justice activism until recently, and only my new line of research in infectious diseases made me step back from it. In summary, we live both in our own world as well as the world around us.

Well, I hope I’ve been able to show you that our stereotypes about mathematicians are not all justified, though many are grounded in reality. The one thing mathematicians are usually not, however, is boring. Oh, and that date I mentioned at the beginning? She actually became my first girlfriend. It might be a lucky coincidence that I actually shared a lot of these stereotypes at the time, since I might have reacted quite differently to her comment otherwise. Then again, if I had already been so enlightened back then, we would probably have chosen to see a different movie, like “Rites of Love and Math” – just kidding!