I have a couple of simple passive boxes built with DACT six channel attenuators for this purpose. They sound pretty good, better than just about any commercial unit I've been able to compare.
In my situations, they sound even better with buffer amps after the attenuators (so I can't really call them passives any more) but you might be able to get away without buffers if everything is located close together and you pick your impedances carefully. But it would be more fool-proof with buffers. Although I subscribe to the thinking that fewer stages are generally better than more stages, this is one area where the additional stage is often an improvement. The complexity starts to add up, but that's what happens when we dip into surround.
Paul Stubblebine
Paul Stubblebine Mastering, San Francisco
The Tape Project, LLC
serious student of the audio arts