When SETs first began to be known in the USA, there was widespread awareness the you needed very efficient speakers. As it became more popular, this point has been less emphasized, but it's still true.
A couple decades ago I compiled some statistics from my back issues of Stereophile magazine, and concluded that they consistently recommended a certain loudness level capability when they reviewed a speaker. For a 3.5 watt amplifier, that corresponds to a speaker sensitivity of 97dB. I usually say that means it will satisfy "most audiophiles, most of the time." However, there is a wide variation, depending on the room, the music, and the listener, so this estimate is not always an accurate guide.
Most modern speakers run 85-90dB sensitivity, and would want 50-15 watts per channel respectively. Your A-25s are 90dB, and would want 16 watts by this calculation. Speakers from the golden age of tube audio - say, 1950-1970 - however often were 95-100dB, which is ten times as sensitive. These tend to be quite large, and often use horn tweeters. Classic examples are Altec A-7s and Klipsch Cornwalls, both of which have long been associated with SET amplifiers. The other fairly widely popular approach is the fullrange single driver crowd, with Lowther and Fostex being iconic examples. These days there is more variety available, but speakers for use with SETs are still a very small minority.
So, that's the story (or at least, my version of it) on speakers.
If you were to keep your A-25s, it's possible that a bit more gain would make it loud enough, but it is equally possible that it would overload the amp, creating too much distortion. For that reason I can't recommend spending a lot on that possibility until you test it. The easy test would be to connect the PAT-4 outputs to the Stereomour inputs. You can use either the PAT-4 phono stage or the Reduction as input to the PAT-4. This should give you enough gain to drive the Stereomour to and beyond its linear range, so you can see the effect of greater drive.