This was causing state pollution in the new selection. I can't see why
this needs to happen when a selection changes to another.
This fixes https://github.com/ppy/osu/issues/30839 and also the same
issue happening for the new combo toggle.
Tests all seem to pass, and I can't immediately find anything broken,
but YMMV.