Yes, there is a difference. It has to do with the noise floor. If the preamp has an audible noise floor then you'd want to attenuate its output.
The ideal is to get each component to work at its best signal level, steering between noise floor and overload (distortion or clipping). That's complicated by the desire to get the primary volume control operating in its sweet spot, generally the top 2/3 of its range. The best compromise will change a bit if you use headphones of different voltage sensitivity.
Of course you may have no detectable problems, in which case you can substitute a resistor for the potentiometer as planned, no problem. That's why I suggested trying it before removing the control on the Crack.