I don't know how to reproduce this issue in a test, so no tests for now.
Nonetheless, this fixes the issue for me at least on one map:
https://osu.ppy.sh/beatmapsets/1899665#taiko/3915653
This workaround is similar to #16475 (the test from that commit got
eventually removed for some reason).