Do Atheists Borrow From the Christian Worldview?

Consider this parable:
A certain mathematician, in a philosophical mood one day, wondered what grounded his mathematics. The math works, of course, but he wonders if he’s missing something foundational.
He consults a friend of his, a theologian. The theologian knows almost nothing about mathematics, but he knows his Christianity.
The mathematician says, “Mathematics is like an inverted triangle with the most advanced math along the wide top edge. The top layer is grounded on the math below it, which is grounded on what is below, and so on through the layers, down to arithmetic and logic at the point at the bottom. And that’s where it stops.”
The theologian nodded his head wisely. “I see the problem—what does the bottom rest on?”
The mathematician was silent.
“In your view, it rests on nothing,” said the theologian. “It just sits there in midair. But the problem is easily resolved—mathematics and logic comes from God. There’s your grounding.”
“Are you saying that I need to convert to Christianity to be a mathematician?”
“No, just realize that you are borrowing from the Christian worldview every time you make a computation or write an equation.”
Satisfied that this nagging problem has been resolved, the mathematician returns to his work and thinks no more of it.
The End.
So, is the mathematician any better off? Is he faster or more accurate or more creative? Do his proofs work now where they hadn’t before? In short, did he get anything of value from the whole episode? Not at all.
And note, of course, that the axioms at the bottom of the triangle aren’t taken on faith, they’re tested. “1 + 1 = 2” has worked on everything so far, but we’ll take notice if we find a situation where it doesn’t. Some mathematical claims are proven and some are tested, but each is reliable.
I’ve heard this “grounding” or “atheists borrow from the Christian worldview” idea many times, but I’ve yet to discover what this missing thing is that is being borrowed.
If we imagine that 1 + 1 equals 2 only because God says so, that means that a universe is possible where 1 + 1 doesn’t equal 2. That’s a remarkable claim, and I’d like to see it supported by the theologian rather than simply asserted without evidence.
“God did it” is nothing more than a restatement of the problem. “God did it” is precisely as useful as “logic and arithmetic are simply properties of our reality” or “that’s just the way it is” or even “I don’t know.” An interesting question has been suppressed, not resolved. In fact, by the theologian’s own logic, his answer rests in midair because he provides no reason to conclude that God exists. His claim is no more believable than that of any other religion—that is, not at all.
The person who stops at “God did it” has stated an opinion only—an opinion with no evidence to back it up. It doesn’t advance the cause of truth one bit.
Mathematics is tested, and it works. Scratch your head about what grounds it if you want, but God is an unnecessary and unedifying addition to the mix.

God is an ever-receding pocket
of scientific ignorance.
— Neil DeGrasse-Tyson

(This is a modified version of a post originally published 11/25/11.)