As i suspected the PC is injecting the noise over the USB bus or ground. As to how to best cure it i cant really say, but here are a few things to try.
-It could be poor filtering in the PC power supply, rather than replacing it try connecting it to a filtering multiblock or UPS, and connect the amp/dac to the mains the other side of it. Idea being to create that barrier between them. For example in my setup i have a two way multiblock from the wall, one socket goes to a UPS that feeds the PC, Monitor, Cell phone chargers are other misc supplies. The other socket goes through a mains filter box which feeds all the audio equipment, DAC's, Amps etc.
-Try different USB ports, some motherboard will have multiple ports, and sometimes from different controllers so you may find some are noisier than others. In my experience USB3 interface's are generally quieter and better quality.
-USB Isolators, these will break the connection between the PC's USB Power and chassis ground of the computer so when used with a dedicated power supply there is no path for the noisy PC power supply to reach the DAC. I built an Isolator box with a Linear power supply to do exactly that. No guarantees it will fix your problem, but i can say my Isolator made a huge difference for me. In my case the DAC is USB powered so the cleaner USB supply was a benefit in itself.
Here is a nice little all in one kit currently being developed that should be available any day now
https://www.head-fi.org/t/703334/doodlebug-usb-isolator , by chance i just built and tested a pre production prototype of that board last night and it performs just as good as the unit i built myself. As far as i know he plans to sell them assembled in a box ready to go, all you need to do is connect a 9v DC power source to it and away you go.
@Jamie, buy yourself a length of Belden 19364 shielded cable and make a power cord. It has a foil outer shield so almost completely kills EMI radiation. I have an EMI detector which is great for finding power cables inside walls etc, i can touch it up against that cable and it still cant see it.