There are several different answers to this question that depend a bit on the headphones you're starting with.
The easiest answer would be for those who own something like a Hifiman HE-6 or an AKG K-1000. These are very power hungry headphones and not particularly high impedance either. In this case, the amount of power that can be sourced from something like a phone is severely insufficient.
When you mention the HD-650's and the 103dB/mW sensitivity, it is actually 103dB@1V RMS, which is about 3mW at 300 Ohms. When you try to get this out of an iPhone 6 for example, that's all that's there, once you exceed 3mW, you enter hard clipping. This will be especially audible when listening to music with decent low frequency content at anything other than a moderate volume.
Tube headphone amps don't exhibit that behavior, and in the case of the SEX and Crack, you have so much headroom that this is no longer concerning.
Luckily for you, you can easily make the comparison on your own once your SEX is built