But that’s my point, this is impossible, mathematically speaking. For any theory that only works if the universe of discourse is infinite, if we take any finite part of said universe, we can create a different theory that behaves exactly as the infinite theory for the finite part, but does not require the rest of the infinite domain.
For example, if we take a theory like Peano Arithmetic, which requires an infinite universe since every natural number has a successor (which is different from all previous numbers), and we take a finite set of examples to check, say, we check that the number 10 exists, we check that it has a successor 11, we check that 5 * 8 is defined, yup, it does, it’s the number 40, we check that 2+3 is equal to 3+2, yes, once again seems to check out, then from only these examples we cannot conclude that we are working with the natural numbers: it might be that we’re working with the numbers 1 to 50, and just never bothered to check whether 50 has a successor.
Our theory may be requiring the universe to be infinite, but we cannot empirically determine that it is the correct theory, and not some other theory that works equally well, but only on a (large enough) finite part of the universe that the original theory requires.
This isn’t a constraint on humans, or on our physics, it’s a mathematical / logical constraint: it’s impossible to verify empirically whether something is infinite.
Nowhere in nature is it necessary that a theory is clean or beautiful. Quite the contrary, actually, there are a lot of things that appear to be true on first glance, but actually turn out to be a lot more ugly than expected. There exist extremely simple questions that can have incredibly complicated answers (think of Fermat’s last theorem, or the four-colour theorem).
Even proving consistency of a theory is something ugly: mathematics is either inconsistent (i.e. contains a contradiction), is relatively trivial and useless (i.e. cannot assume that all natural numbers exist) or cannot prove that it is correct and will always contain unprovable statements (by the incompleteness theorems).
There’s really no reason to assume that the truth is clean and elegant.