What Doc B and PB said - your scope is triggering on non-audible, super high frequency stuff from computer, routers, wireless phones, etc.
It's been a few months since I played around with my scope, and I'm barely past the stage of just knowing how to turn it on/off and hook up the probes, so I might be a bit off about what follows. But it may help.
i think you need to increase the horizontal time divisions A LOT. From your pics it looks like you had the time division set to 4 ms - I think that's too small of a window to show and trigger on a 60 Hz waveform. Try 100 or 50 Ms instead. Or, to get some practice dialing it in visually/manually, here's what I suggest for searching for 60 Hz hum. Use both probes/channels. Set the switch on probe/channel 1 to 10X and hook it up to the hot wire of your incoming AC voltage on the IEC inlet. Make sure you have probe 1 set for 10X on the scope. Dial in the horizontal divisions until you see a nice 60 Hz waveform. If it's moving all over the place, that means the scope isn't auto-triggering or triggering on the wrong thing. Either hit auto-trigger, manually select a trigger (e.g., falling or rising slope), or manually adjust the trigger line so that it falls within the signal image - any of these should give you a stable waveform on the screen. Once you have that, you know how to dial in the horizontal time divisions for probe/channel 2. Set the switch on probe 2 to 1 X and do the same on the scope. Hook up probe 2 to either the left or right rca output jack (NOT the power supply, b/c that might damage the probe/scope depending on how much voltage they can withstand). Dial in the horizontal divisions to match the figure you used for probe 1. At the this point the easiest way to see what's going on at the rca output is to turn off the display of probe 1, so only probe 2 is visible. Adjust the vertical divisions until you see a waveform (assuming there is a 60 Hz waveform). Then you can measure it (and I suspect your scope can do rms measurements for you - on my scope it's found under a menu intuitively named "measure"). There's also a way to keep both probe traces on screen by shifting them away from each other using the vertical position adjustment- but in your case that doesn't serve any testing purpose (although it looks kinda cool and is worth figuring out how to do for when you want to compare simultaneous waveforms).
Hope this helps, Derek