Unless you have the equipment to be scientific about it, it is black magic. I have bypassed both power supply and signal path caps. The KK Teflon Signal path bypass in my Ack! dAck! remains. So does the KK Teflon bypassing my Parafeed caps in my Paramours.
I have three different kinds of caps as an additional final PDMPS stage in my FP 2. There is a KK Teflon at the top, of course.
Weeks, maybe months, after building my Eros I experimented with different caps at the output. At Paul Joppa's urging I tried both one and two KK Teflon 0.22uF caps here. The single won over the higher value. I could hear no bass lost using a 0.22uF here rather than the designed 0.47uF, and I was using my sub.
So to me it is a try it and see kind of situation. You could buy a nice 0.1uF and alligator clip jumper it into several places just to hear the results.
Enjoy this, it is a hobby!