I listen to the same type of music as you (i.e., classic rock and jazz, though mostly pre-fusion for the latter), and recently sold a house where I had the amps in a similar-sized room with wood floors. The Klipsch Heresy III's I had built in late 2009 are rated 99dB @ 1W/1m, which is exactly what "Joppa's Rule" suggests is able to provide reasonable loudness from just two watts. Needless to say, I rarely had to use more than a quarter to half turn on the volume pot with the Klipsches! Unless you're keen on blasting out the windows, I'd say two watts on 99dB speakers is more volume than you may ever need. I'd be surprised if I ever use more than half of a watt, actually. Now it's more like 1/32W in a thin-walled NJ apartment. My Klipsches will have to be patient until I finish my post-doc and can move to a more music-friendly home!
Prior to having the Heresys built, I used a pair of $350 Polk RtI A1 speakers, which received a great review in Stereophile when they were released. These were only rated as 89dB/W/m, but more than sufficed for near-field listening from a couch just a meter or two away. They would not have been able to fill a room with sound on 2W, but got surprisingly loud and had great bass for such small (5") woofers. The SEX really is capable of driving a lot of speakers, provided that you don't expect stadium levels of loudness with anything less than horns. I should mention that the newer iteration of the SEX amp, which is the one you are considering to build, is supposed to have much better transformers than the version 2.0, which used Speco 70V transformers as autoformers. Those of us with the older version of the amp often have better irons built by MagneQuest, which is what I'm waiting for as we speak. So, the updated amp may actually be even more tolerant of off-the-shelf speakers and provide better bass than the auto former version I built in Feb. '09.