Nothing more than a language? Do you really believe that math is on the same footing as English? I think you're being disingenuous for dramatic effect.Math is nothing more than one of many languages we use to describe our existence as we see it.
Math indeed has a language: we agree to certain conventions of symbols and syntax, which must be learned in order to communicate mathematical ideas. But the language is arbitrary and has nothing to do with math itself. We could (and have) changed the language and still retain all the meaning. I point at Fluffy and say "dog"; my German friend says "Hund", but we're both talking about the same thing, independent of the language chosen. Math is fundamentally about that "thing in itself", irrespective of the language used.
The "thing in itself" that is the subject of math is pure logical thought, with all the irrelevant details abstracted away. As such, math comes equipped with an enormous amount of machinery for describing, predicting, and verifying the relationships between anything and everything. In other words, math can not only describe this universe, but any other universe. That's pretty wonderful to me. Now, you may say "so what?" -- if math is so wonderfully generic that it can describe any and all universes, then of course it can describe this particular universe. Duh, right? Indeed, but that's missing the point that there even exists such a universal (multiversal?) system of logic, which we somehow have access to. Out of all human inventions, none come even remotely close to mathematics; I mean, try to visualize where we'd be without it! Given its unimaginably vast richness (of which we've so far explored only an infinitesimal fraction), it seems more than plausible that we didn't actually invent math, but discovered it.It's no wonder that the math we have developed to describe the universe, well, describes it.
You say that like it's a bad thing, but that's how formal systems work -- use axioms and rules of inference to derive theorems. Remember, you're free to choose any axioms you like; math is not about the axioms, it's about the interesting things that happen after you've chosen a set of axioms.It's axiomatic.