Acute observation that half the filament was dead on one tube! That would indeed create asymmetric performance of the tube and make it harder to balance any hum.
"..when not turned on I read 1.8mV..."
You may have a magnetic field that is interfering with measurements, or that the output transformer is picking up, or both.
First, read the AC voltage with the meter leads shorted to each other. If this is not near zero (less than 0.2mV, I would say) then twist the meter leads together and try again. If the twisting helps, then there is a big magnetic field which is intercepted by the wire loop made up of the meter leads - the twisting reduces the area enclosed by the loop. Leave them twisted, and maybe even tape them together every foot or so.
Once you are able to measure near zero AC voltage with the leads shorted, then try the amp's outputs again. If there is still 1 or 2 mV at the output with the amp turned off, then the output transformer is picking up a field. The usual and most likely source is the power transformer on some other piece of gear, but old knob-and-tube wiring in the house can also generate large fields, or appliances, certain lighting fixtures, etc. You may find that moving or re-orienting the amps will help.
Once you can measure near zero output from the amp when it is off, then you can assess what noise is being added by the tubes and power supply.
Sorry that this seems to be getting complicated.
