It's important to note, you don't have to actually believe modern physics.
When they're talking about the here and now they're amazingly accurate. For instance they know how much heat will be released burning a barrel of oil, and how rainbows work, and so on.
But when it's about events 14 billion years ago, when the entire universe fit in a space smaller than a grain of sand, you have to take it all with a grain of salt.
jj2007: Dumb question from a mere mortal: that "extra space" expands against what? Non-space? Matter? Vacuum? When looking at "the universe", aren't we rather looking at a sphere in infinite space, i.e. a ball filled with matter, more or less densely, whirling in a bigger area that has absolutely no trace of matter?
- General Relativity specifically assumes there is NO thing outside - not space, time, matter, vacuum or even "non-space".
That's partly what makes the math difficult. GR doesn't assume a larger space we're "embedded" in. The point of differential geometry is to describe timespace "from the inside", from local curvature only. Then you piece together different local curvatures to get the total timespace. You may have heard of "Christoffel coefficients": that's what they're for.
The details don't matter (unless anyone is interested), I'm just emphasizing that in GR there is, deliberately, NO "super-timespace" we're embedded in. As the universe expands, it creates the time and space to expand into.
Note, GR doesn't insist there's no super-space, only that it's not necessary for GR's purposes. But, if GR's right, that super-timespace has to be a different time and space continuum. It can't just be more of the same old time prior to the Big Bang, and more of our type of space to expand into. For instance in the simulation theory it's the timespace inhabited by the programmers; our "lesser" timespace is just an artifact of the program (somehow).
OTOH If GR's wrong (entirely possible) then the Big Bang (if indeed it ever happened) could have been just a huge explosion, more or less as you imagine. Time could go back forever before the event, and "infinite" space could have always been there ready for the explosion to expand into.
Don't forget the whole thing might be an illusion anyway! It's good to understand what modern physics says about this issue, but it's not worth too much cogitation; the answer simply is not available.