The "un-filter" step causes a `SelectNextRandom` invocation. If this happens to select a difficulty in set 3 other than the previously buffered difficulty #2, the subsequent test would fail.
I've split this test out to remove the random element, but added a new assert to ensure buffered (previously visited?) difficulty is re-selected on return to the same set.