Audio Note pot is 6mm, so the original knob will fit (but you can't put a 6mm knob on the 1/4 inch pot).
Best way to do it is 1 turn of Kapton (or any tape you are happy to use) tape around the shaft. Keeps the knob centered (It will work anyway, but this is better)
Similar to what you describe, I didn't notice much difference in sound however low end balance is much better. Also it has a *much* more premium feel to it when turning. Neither has scratchy sound in the audio path like some really low end pots