The frozen balloon shrank because the average kinetic energy of the gas molecules in a balloon decreases when the temperature decreases. This makes the molecules move more slowly and have less frequent and weaker collisions with the inside wall of the balloon, which causes the balloon to shrink a little.
