Power shouldn't have anything to do with it as long as it is capable of reaching the same listening volume without clipping as your dedicated Stax amp. If you are running the SEX wide open to achieve listenable volume then you will have a higher level of distortion than if the amp as idling along. That box is in essence a pair of PP transformers wired in reserve with a bias suppy. It acts as a phase splitter and a voltage step up. By putting a more powerful amp in front of it you can swing higher voltages and reach higher volumes. That box is the limiting factor if you follow the train of thought.
I have nothing against the tube based Stax amps but to my ears, I didn't care for any of them. My SEX is still being tweaked to my listening preferences and that is something you can't do with the Stax amps. It's a hell of a lot simpler having one and if they fit your listening preferences then congrats!
If you like the sound of the transformer box with other amps you can easily build one yourself with much better iron. A 5k transformer has the ideal winding ratio but others have had good results with 8.6k too. It's a balance between power output of the amp and finding a transformer with the lowest turn ratio to achieve the voltage step up to drive your ear speakers. If you search on Head-Fi there are a few old treads on the topic. I was going to go this way but again, it's a compromise. The SEX offered a simple platform to experiment with I'm happy with the results. To be honest I don't listen to headphones enough to justify a dedicated amp for them. My current focus has been tailoring the SEX to match my speakers and listening preference. Once that's dialed in I'll put on the cans and see if they sound good enough