For headphones which are very sensitive (Grados, IEMs, etc), the low impedance setting will provide a blacker background. It will also provide lower distortion and better damping, especially into low impedance headphones.
The high impedance setting will provide more power out of the amp provided the headphones used are not low impedance headphones. The trade-off here is a slightly higher noise floor and a bit less damping and potentially more distortion. If you are using a 100+ ohm headphone on the high impedance setting, odds are you won't hear the difference in noise, nor the increase in THD and decrease in damping.
If you have super sensitive 16 ohm IEMs, you may very well be able to here these changes more easily.