I think Grainger pretty much nailed it. Gain is not power, and unfortunately there are only gain controls, no power controls. For a given input voltage (from the CD player for instance) more gain results in more power, of course, but then more signal from the CD player also makes more power.
In fact, the audio signal has a very high peak-to-average ratio, and you can easily find recordings where the instantaneous power - let's say, averaged over 0.0001 seconds, right when the stick strikes the cymbal, is 25 watts yet the moving average power (over say 0.2 seconds, which is more like what we hear, and what the speaker can be damaged by) is only one watt. In contrast, other recordings (a certain Nine Inch Nails album comes to mind) can have 25 watt instantaneous peaks with 24.5 watt moving average.
The only reasonable answer is, as Grainger has suggested, to trust your own ears, and after the first two martinis don't let your friends adjust the knobs! :^)