The volume pot does not affect this. I can turn it all the way up and it is the same effect over the whole range.
I am testing with MONO recordings (from PC) and it's always there. Also, when I switch the input chords. It is always a bit louder in the left channel.