I would suggest clipping or tacking them in one at a time as the most sensible approach.
If you decide to do the switch I'll suggest that you want the switch between the output end of the cap and the headphone jack, and that the out of circuit cap outputs get grounded when switched out of the circuit, ideally thru something like a 2.7K ohm resistor similar to the one on the headphone output. Make sure all of the cap inputs are attached to the plate voltage so they stay charged or you will get some nasty popping as they charge up when you throw the switch with the amp running, and also subject that poor switch to some gnarly inrush current. Also, no promises on what might happen to your headphones if something goes wrong.
In other words you want the caps to think they are in the live circuit even if they are not.