Good detective work. Yes, this is the dirty ground issue to some extent, but it is most likely due to noise from the computer and it's power supply going back out on the AC line.
You could try an isolation transformer on the computer power cable, or possibly consider finding a lower noise power supply for the computer. Also, sometimes this may improve with my earlier suggestion to tighten all the screws in the computer -- card slot brackets, all structural pieces, disc drives, power supply, case, every single screw you can find. I've seen and heard this work wonders more than once and this may be one of those situations where it could possibly pay off. No real way to tell without doing it though.
-- Jim
Jim Rebman -- recovering audiophile
Equitech balanced power; uRendu, USB processor -> Musette DAC -> 5670 tube buffer -> Finale Audio F138 FFX -> Cain and Cain Abbys near-field).
s.e.x. 2.1 under construction. Want list: Stereomour II
All ICs homemade (speaker and power next)