I started to write an answer, then decided to mull it over a bit.
There are a few reasons why a manufacturer would set a minimum impedance cutoff, so the best I can do for you is to explain why we might run into that with our products and you can extrapolate that to your unnamed headphone amp.
For something like the Crack, there is a limited amount of output current available, and this presents a limit to the output level available into low impedance headphones. Additionally the output impedance is a bit on the high side, so the impedance curve of a low impedance pair of headphones will drag the amplitude at the output of the amp all over the place, and you'll get fat, bloated bass and treble that's obnoxiously present. The Crack is also cap coupled at its output and those caps are sized for headphones that don't upset the balance of the first two points.
For an amp like the SEX amp wired for 4 ohm speakers, the amp is happy with at least a 4 ohm load and that allows an appropriate load to be reflected to the output half of each 6FJ7. As you go below that, gain will decrease a little bit and distortion will go up, though an amp like the SEX amp is pretty tolerant to this. If you found some 2 ohm headphones, they would probably work fine like this, but it's still an illustration of other problems that can creep up.
