The "un-filter" step causes a `SelectNextRandom` invocation. If this happens to select a difficulty in set 3 other than the previously buffered difficulty #2, the subsequent test would fail.
I've split this test out to remove the random element, but added a new assert to ensure buffered (previously visited?) difficulty is re-selected on return to the same set.
When `FrameStabilityContainer` decides it needs multiple updates on the same frame, it ends up with an elapsed time of zero. This was interacting badly with the condition used in `RulesetInputManager` to govern playback direction.
I have changed this to use `Rate` as exposed by the frame stable clock.
- Closes#6198.