Viktor,
It depends on your system. If you use the same source and get good volume control then there is no worry. What Williaty has is a problem that he can only use a very small turn on his volume control. To make small adjustments at low listening levels, or any listening level, the control should, ideally, be at full clockwise rotation with your lowest input and your highest listening level. No one gets that ideal situation.
What most of us have is a good listening level at 12:00 to 3:00. That works well and for late night speaker listening, or say background listening with a headset, you can make very fine adjustments of the volume control and get what you want.