Minute Physics gives an incredibly concise overview of what we know, and what we don't know, about the beginning of the universe. Of course, the most tantalizing part is the point "where we don't know what we're talking about." Perhaps knowledge is like infinity; no matter how much we learn, we also learn how much more there is to learn. -via Viral Viral Videos
As for the the comments from the physics purist and the religion hater: Thanks to you as well.
A singularity is just a place (can be a point, line, sphere, curve...) where some quantity becomes mathematically undefined or infinite. Sometimes this is just due to a bad choice of quantities you look at, like an inadequate coordinate system. Longitude and latitude suffers this problem at the poles, where longitude is undefined. That doesn't mean someone can't stand on the north pole, only that the particular coordinate system fails at that point, and a different one could be used that would work fine there.
There are other kinds of singularities that cannot be fixed by changing coordinates and involve physical quantities becoming undefined or infinite. An example would be the center of a black hole where general relativity says density becomes infinite and there are problems with any coordinate system you use. It is not a term inherently meaning we don't know what is going on, but typically scientists see problems with current theories if you take certain values to extremes, or just presume that there may be more going on at that point, so that such a point would likely involve things beyond current theories.
You could talk about another example: a bouncing ball that with each bounce loses some energy and doesn't bounce as high as the previous bounce. If just considering gravity's pull on the ball and the dissipation of energy at the bounce, it would look like there should be a point in time where the bounces become infinitesimally small and the bounce frequency becomes infinite. This doesn't happen because other dynamics kick in if the bouncing happens too fast, or the distances between the ball and the ground get small enough.
Some proposed cosmology theories remove singularities, while others do not. This leads to the beef with how he pushes that idea that space is infinite. It is useful to consider, but there are also plenty of work that based on theories that have a finite universe, with or without initial singularities, that could still be pictured as the entire universe being within a single point at the beginning.
And I wonder which evidence he was referring to the universe being 20x larger than the observable universe. I've seen estimates much larger than that before, although they are usually all based on some big assumptions, e.g. we see X, but if it was really Y, the universe would need to be this much bigger to make it look like X.