But wouldn't freezing the pants kill a lot of the bacteria?

... yes, the answer is yes.

Few leave mainly because education is typically poor (one room schools, taught up to ~8th grade by young Amish women)... so good luck getting a good job having not graduated from high school (and good luck getting into college after that).
Also, many families don't help children financially if they don't stay in the sect. So basically when they leave the church they leave their family, and they start out in modern America that thinks they're weird with an 8th grade education, strong (but old-school) vocational trade skills, but little knowledge of technology. So, not terribly hard to see how the church could be a pretty comfortable choice.