John,
There is a rather large physical size difference over the whole range. With each jump up in the size range, because the collets are physically larger, then they can have larger holes in them. So the higher the number, the more collets in the set.
I think the range goes from ER16 which is minute when compared to an ER40. It is so that you can have a range that also matches the machines they are designed to fit. The ER40 would be more suited to the Bridgeport and above sized machines (which can handle much larger cutters), whereas the ER16 would go well on a micro mill, which can only handle the smaller sized cutters. I think the ER32 size has become the standard to be fitted to the X2 and X3 sized machines.
The other main advantage of the ER system is that each collet has a full 1mm compression range. So even though they are marked up in mm they can also hold imperial cutters just as well.
John