Very enlightening to read that, thank you. But this is what I mean, what does it make you think, or feel, to know that everything we're doing could (potentially, in theory) be simplified in some way?

Doesn't bother me in the slightest. I'm sure that one day we'll understand the universe much better than we do now, and our equations and simplifications will look silly. "They could have invented faster-than-light travel so much faster if only they understood...."

But we do the best with what we know now, and I think we're doing pretty well.

However, and the big, if no doubt stupid, question: if mathematics is correct in this manner, why do scientists need to speculate, measure things, iterate it a billion times etc.?

Are we to decouple the hypotheses of science and the underlying maths of a particular theory? I like that, technically in this way, if science is wrong, mathematicians can just hold up their hands and say "not our fault"

Math and science are definitely independent areas of study. They rely on different forms of reasoning (induction vs deduction), and the result is that mathematics and logic are on significantly stronger ground than science - not that science is on shaky ground.

Scientists concern themselves with an understanding of the universe that mathematicians and logicians never concern themselves with. Scientists attempt to understand the nature of everything. Mathematicians limit themselves to the understanding of mathematics itself. That's why scientists are constantly hypothesizing, testing, measuring, coming up new hypotheses, and eventually adopting theories. Mathematicians simply derive.

So, indeed, science can be entirely wrong while the underlying mathematics remains correct. A scientist could hypothesize that flies spring forth naturally and spontaneously from unattended meat. He could derive equations that model the rapidly expanding fly population from an unattended meat. He could even show that when meat is left unattended, these equations accurately predict the fly population to within fantastic accuracy. He can still be wrong about the nature of the observation though, flies do not spring forth spontaneously - no matter how correct the mathematics were that modeled their population.

Anyway, I know there have been famous examples of people who have found the seemingly unreasonable power of logic (etc.) to be inspiring in various ways. Granted, the vast majority of people probably won't blink an eye, which matches my experience with the scientists I've met, too. Perhaps "many" was my mistake, or maybe it was replying to someone who mentioned "God".

I'm going to invoke Hawking again. Yes, there have been physicists that have found themselves in awe of how well suited our universe is for comprehensible laws of physics and stability. They observe that constants of the universe could be off by only a tiny fraction, and suddenly the universe collapses on itself and explodes within the first moments of its existence. How then did we happen upon the exact set of correct values of these seemingly arbitrary constants of reality. It seems awfully carefully laid out. Many physicists in the past have found this eerie and unsettling. Some of them have found it to be proof of god.

Most of the people I was talking about are not this type of scientist. I was speaking of the engineers and scientists that I work closely with, who concern themselves more with the thermal properties of a solar array, or a voltage spike, or the gravitational field of a lumpy body... and whether this gravitational field indicates the presence of a solid core within the body etc. etc. Only cosmologists struggle with questions like, what would happen to our solar system if we changed the gravitational constant of... gravity by a fraction. I don't work with cosmologists, so I don't work with people who gaze deeply enough into the underpinnings of reality to find precariousness and precision in its structure. The engineers, physicists, and scientists that I work with tend not to be spiritual about reality.

So I should get to Hawking. Hawking's most recent book attempts to lay out a framework for how our universe might have gotten so perfectly balanced. He hypothesizes that when our universe initially formed, other universes also formed... a nearly infinite number of them. Most of the universes that formed had unstable laws of physics or physical constants. These universes collapsed almost immediately. Only those universes whose laws of physics were suitable for stability continued to exist, and only the universes who's physical laws were conducive to life would eventually form life. And so our existence is a product of an enormous sample size. We formed in the universe that is conducive to our existence because that's the only place where we could form. I find this to be much more believable than any sort of mysticism. Cosmologists have needed to postulate that alternative universes must exist with alternative laws of physics. This is a natural and straightforward result of that.