Good points. Two relevant things are generally grounded - the case and core of the transformer, and the secondary windings. The secondary is interleaved with the primary so the primary is adjacent to grounds at several places.
Those points are grounded primarily for safety reasons, and the core secondarily to help control hum pickup.
I know, that's not much detail, but this is a forum post, not a text book! I can recommend the discussion of transformer design in the Radiotron Designer's Handbook 4th edition, aka RDH4 or "the red bible", for much more information and details.