When you have very, very sensitive speakers like the 604D and a high gain amp like the Deuce, you are asking for problems with hair trigger volume. This means that when you turn up the volume pot on your preamp just a tiny bit, you will immediately be at full listening level.
It is very common for people in this situation to blame the preamp for this issue (my preamp has too much gain is the usual complaint), but in reality your speakers can't make use of more than 4-5V or so of signal coming out of the amp, and the Deuce will produce that level with a few tenths of a volt of signal at the input, so your preamp needs at most 0dB of gain to function properly under these conditions.
If you switch your speakers out to some Magnepans, life would be peachy without any issues.
A pair of -10dB Rothwell attenuators plugged into the inputs of the power amp would likely make all this work reasonably well. Running those Altecs off the 4 ohm taps on the amp is also not a bad idea. Most SET amps also have lower overall gain, and this helps a lot with sensitive speakers.
