I don't usually butt in on this kind of thing. But we spent a lot of time last year trying a lot of different volume controls, and found that they influenced the sound waaaay more than we thought possible. As character flawed as the sweetest whispers was in terms of potentially noisy switches and coarse steps, it always sounded better - tighter and thus deeper sounding bass, better resolution, cleaner treble - than the pots we would try. And it inspired the BeeQuiet, in which we accepted that it will be a bit more expensive to take out those last sticky issues and thus added more expensive parts and more complexity, and which I like better than anything else I have tried so far.
SW certainly wasn't perfect, and we made an effort to explain why it was compromised in terms of parts quality so that we could design it to hit a price point. Last year's experiments in looking for an all out volume control put it ahead of all pots we tried with the closest runner up being an Alps Blue that we aren't even sure was a legitimate one. And SW came out ahead of other stepped attenuators we tried too. So it may not be the most convenient thing to open up a noisy switch and futz with it. And you may find that an Alps Blue or Goldpoint or whatever sounds better to you. They are really nice! But my experience indicates that it might be worth it to try getting the SW back up to snuff first.