The 0.1uF cap on the shunt regulator, along with it's associated resistor, stabilizes the feedback loop to prevent oscillation and/or ringing. Don't change the values! It's been a long time since I did the analysis, but if I recall correctly the instability happens at around a megaherz, which would lead sometimes to very hard-to-trace, unpredictable effects.
If you do change the cap, use something physically smaller to minimize RF sensitivity. Don't use Mylar without re-doing the stability analysis because Mylar develops a significant ESR at radio frequencies. Maybe something like mica, or NPO (C0G) ceramic.