Since I study neither gender issues nor genetics, and since I am not a scientist (insofar as that is different from being a mathematician), the only "authority" I can bring to the question is in the area of mathematical ability, so that's what I'll stick to here.
As it happens, the question is fairly closely related to the topic of that new book that's got me onto the media circuit in the first place, namely the innate mathematical abilities of various living creatures, including - but not exclusively - humans. (For a brief overview, see The Math Instinct website at www.mathinstinct.com.)
The first thing to observe is that the very question "Are men on average better than women at math?" is not sufficiently precise. Expressed that way, the question implicitly assumes that mathematical ability is a single mental capacity. It's not.
"Doing math" involves all kinds of mental capacities: numerical reasoning, quantitative reasoning, linguistic reasoning, symbolic reasoning, spatial reasoning, logical reasoning, diagrammatic reasoning, reasoning about causality, the ability to handle abstractions, and maybe some others I have overlooked. And for success, all those need to be topped off with a dose of raw creativity and a desire - for some of us an inner need - to pursue the subject and do well at it.
I guess it's possible that if you have any one of those specific cognitive abilities I just listed, then you have them all in equal amounts, but frankly I doubt it. In fact, based on oft-repeated claims about the relative mental strengths of men and women when it comes to things like linguistic ability and logical reasoning ability, my list contains some elements often attributed to women and others to men. Whether those claims have any real substance is another matter. But even if they do, it looks to me on the face of it that any differences might pretty well balance out when it comes to doing math.
Still, whether there are aggregate innate gender differences in mathematical ability between men and women is, of course, an entirely empirical question, that could, I suppose, be resolved by doing sufficient research. My own feelings about the entire Summers episode is that much of the debate obscured what I think is a far more important point: that there are plenty of well catalogued data pointing to the enormous effect on math achievement brought about by social and cultural influences and by the often subconscious attitudes teachers bring into the classroom. Those factors influence math performance both across racial and social groups as well as gender. And we can try to do something about it right now. Indeed, many people are doing just that. My money goes on this being the pressure point where a greater effect by far can be achieved.
But that is not what I want to discuss today. I want to go back to that question of what it means to do mathematics. That might seem to be a pretty simple question, but unless you take a particularly dogmatic stance on the matter, it turns out to be extraordinarily difficult to come up with an answer.
One such dogmatic stance is the one I myself was brought up with when I was indoctrinated into the world of professional mathematics in the late 1960s and early 1970s. Namely, that mathematics - and here I am thinking about pure mathematics, which is what I was taught - is about developing rigorous logical proofs about formally defined abstract structures, starting with a set of precisely formulated specific axioms. In short, Hilbert stuff, right down to an acceptance of his famous "tables and beer mugs" remark. I took to it like water (or maybe beer), and it provided me with immense intellectual satisfaction and a great career for twenty years or more. For me, anything that did not fit that Hilbertian mold simply wasn't (real) mathematics.
But after a couple of decades of that, my interests began to broaden, and I shifted my focus to trying to apply mathematics - in my case not to the physical, business, or economic worlds, as many of my colleagues were starting to do (I guess it was an age thing, where there is a tendency for both your intellectual interests and your butt to broaden), but to the world of linguistics and social science. I also started to step back from the activity of doing math (either pure or applied) to reflecting on what it really means to "do math." One outcome of that change in perspective was my book The Math Gene, published in 2000.
The main question considered in The Math Gene was how did the human brain acquire the capacity to think mathematically? To answer that question, I had first to analyze just what "think mathematically" means. Since mathematics is a Johnny-come-lately in the human cognitive field, doing math has to come down to taking mental capacities that were acquired by our ancestors over tens or hundreds of thousands of years of evolution, long before what we usually think of as mathematics came along, and using them in a novel way to perform a new trick (or set of tricks).
The thesis I eventually came up with suggested that doing mathematics makes use of nine basic mental abilities that our ancestors developed thousands, and in some cases millions, of years ago, to survive in a sometimes hostile world. Those nine mental capacities are:
1. Number sense. This includes, for instance, the ability to recognize the difference between one object, a collection of two objects, and a collection of three objects, and to recognize that a collection of three objects has more members than a collection of two. Number sense is not something we learn. Child psychologists have demonstrated conclusively that we are all born with number sense.
2. Numerical ability. This involves counting and understanding numbers as abstract entities. Early methods of counting, by making notches in sticks or bones, go back at least 30,000 years. The Sumerians are the first people we know of who used abstract numbers: Between 8000 and 3000 B.C. they inscribed symbols for numbers on clay tablets.
3. Spatial-reasoning ability. This includes the ability to recognize shapes and to judge distances, both of which have obvious survival value for many animals.
4. A sense of cause and effect. Much of mathematics depends on "if this, then that" reasoning, an abstract form of thinking about causes and their effects.
5. The ability to construct and follow a causal chain of facts or events. A mathematical proof is a highly abstract version of a causal chain of facts.
6. Algorithmic ability. This is an abstract version of the fifth ability on this list.
7. The ability to handle abstraction. Humans developed the capacity to think about abstract entities along with our acquisition of language, between 75,000 and 200,000 years ago.
8. Logical-reasoning ability. The ability to construct and follow a step-by-step logical argument is another abstract version of item 5.
9. Relational-reasoning ability. This involves recognizing how things (and people) are related to each other, and being able to reason about those relationships. Much of mathematics deals with relationships among abstract objects.
All nine capacities are basic mental attributes important to our daily lives. The human brain had acquired them all by 75,000 years ago at the latest.
A question that arose while I was writing The Math Gene, but which I did not pursue very far at the time, was to what extent those nine mental capacities could be found in creatures other than ourselves? To put it bluntly: is doing math a uniquely human activity?
This is where you need to decide what you will accept as "doing math." Hilbertian purity aside, if your understanding of doing mathematics entails deliberative, purposeful, self-aware, directed, thought about properties and questions concerning numbers, geometric shapes, equations, etc., often (but not necessarily and not always) carried out with the aid of a paper and pencil, then indeed mathematics is a uniquely human activity. But if you do want to make that your definition of doing mathematics, then you should be consistent.
For example, when we use a calculator or a computer to solve a math problem, would you say we are still doing mathematics? Surely yes. But in many cases, the calculator (especially if it is a fancy graphing calculator) or computer (especially if it is running software such as Mathematica) is performing crucial steps that the user initiates but does not actively control, may not be aware of, and may not understand. In such a situation, when pressed, most of us would say that the calculator or the computer "does the math."
What then if some non-human living creature does something similar? For example, every year, many varieties of birds, fish, and other creatures migrate over hundreds or thousands of miles. In human terms, that necessarily involves solving some fairly advanced problems of navigation, including accurate time keeping, speed measurement, orientation, and trigonometry. Now, no one, I think, would claim that a migrating creature is solving mathematical problems the way humans do. Indeed, it seems highly likely that the creature does not even have any reflective awareness of what it is doing. Most probably it is simply following an innate instinct.
But then, your hand calculator or your computer is not aware of doing anything either. In its own terms (if that has any meaning) it is simply obeying the laws of physics, particularly as they pertain to the flow of electrons. Describing its behavior as "doing math" is to describe it in human terms. The computation is in the eyes of the beholder, not intrinsic to the device itself.
Given that your electronic device has far less conscious experience than any bird or fish, if you are prepared to describe actions of your calculator or computer as "doing math," why not describe the analogous activities of a migrating creature the same way?
Of course, you might then counter, "Ah, but the calculator or computer was designed by human engineers to do mathematics." To which I would retort, "But millions of years of evolution by natural selection - a remarkably effective "design process" if ever there was one - is what resulted in that creature being able to do that particular mathematics (i.e., that activity that in human terms amounts to doing math) that is important to its survival.
Once you start down this path of trying to figure out what it means to "do math," you pretty soon come into some deep philosophical issues. Let's suppose we have agreed that computers and migrating birds and fish "do math." Would you say that when a river flows it solves the Navier-Stokes equations (the partial differential equations that describe the way fluids flow - which human mathematicians do not know how to solve, incidentally)? In a sense, the answer has to be yes. But personally I would draw my line - and it's of necessity going to be a fuzzy line since we are talking about descriptions here - to exclude the river from the "does math" category. Follow that path and everything could be said to be doing math, and the concept becomes meaningless. (Unless you are a physicist and you want to view the entire universe as computation, but that's another issue.) To me, to be classified as "doing math," an action by some entity has to lead to a meaningful output. The (loose) definition of "doing math" I take in The Math Instinct is that the activity, or a closely similar one, when performed by a human, would be said to involve doing mathematics. Even so, there are some borderline cases, some of which I consider briefly in the book. (Beavers building dams and spiders constructing webs are two such.)
The problem with approaching mathematics as a purely human endeavor is that we focus almost exclusively on the conscious performance of computational processes - numerical, algebraic, geometric, etc. - often carried out with the aid of a pencil and paper, or these days some form of electronic computational device such as a calculator or computer. Those kinds of activity are certainly part of mathematics, but if you start from the fact that mathematics is about recognizing and manipulating patterns, then viewing the paper-and-pencil stuff we humans do as being all there is to mathematics is like saying that flying is about having wings and flapping them up and down. True, that is how birds fly, but if you take that as being what flying is about you exclude all those jet aircraft that fly around the globe every day. Flying is more fundamental than either birds or airplanes; it is about leaving the ground and moving through the air for extended periods of time. Feathered wings that flap and metal wings engineered by Boeing (that hopefully do not flap) are just two particular ways of performing that activity.
Once you view mathematics as the science of patterns, and think of doing math as reasoning about patterns, and allow for different devices or creatures to "reason" in their own way, recognizing that classifying a particular activity as "computing" or "doing math" is in the eyes of the beholder, you realize that, far from being unique to modern humans, mathematics is all around us. In The Math Instinct, I give many examples of non-human, "natural born mathematicians," some of them quite amazing. To put the matter somewhat anthropomorphically, through the process of evolution by natural selection, nature has endowed many of the creatures around us with built-in mathematical abilities that, from a human perspective, are often truly remarkable. In short, Mother Nature turns out to be one of the greatest mathematicians of all. Continuing in this anthropomorphic vein, I can't resist ending with the observation that she (Mother Nature) is female.