Minie Balls: why so much infection?

Probably not. Just think of all the germs and bacteria on a battle field that troops would have on their unwashed hands when loading the gun. That was probably all that was needed.