The 120 ohm resistor is a standard, IHF in the past and IEC currently. As far as I know, its only purpose is to keep low-impedance phones from being excessively loud compared to high impedance phones. It has little effect on high impedance phones, but greatly decreases the damping factor along with the signal level on low impedance phones. So it imposes an extra constraint on the design of low-impedance phones.
All these standards are from a pre-iPod era. iPods and other small portables have low voltage batteries and can't put much power into a headphone unless the phone impedance is low and the source impedance is very low. In other words, the standards are out of date. More and more phones are low impedance so they can work with low voltages, and many of them are not actually very tolerant of a high impedance source. The standards have become obsolete, and more and more manufacturers are ignoring the extra constraint.
Bottom line, you can yank the resistor if you want, it will probably improve the quality of the sound. If you have very sensitive low-impedance phones they will get too much signal, and even the very quiet SEX amp might make too much hum and/or tube rush noise. In that case, you'll want to keep the resistor, or substitute a suitable low-impedance level control. But in the case of the subject lower-sensitivity phones, yanking the resistor is probably the best sounding option.
The SEX amp is transformer coupled, which reduces the risky startup transients to insignificance. You are right and prudent to ask about it since there are headphone amps which do have such transients, and modifications might exacerbate them.
