There are suspicions that the straight 5s retry could have caused a
situation a few days ago for `osu-server-spectator` wherein it was
getting hammered by constant retry requests. This should make that
a little less likely to happen.
Numbers chosen are arbitrary, but mostly follow stable's bancho retry
intervals because why not. Stable also skips the exponential backoff in
case of errors it considers transient, but I decided not to bother for
now.
Starts off from 3 seconds, then ramps up to up to 2 minutes. Added
stagger factor is 25% of duration, either direction. The stagger factor
helps given that if spectator server is dead, each client has three
separate connections to it which it will retry on (one to each hub).
Closes https://github.com/ppy/osu/issues/26835.
I must have not re-tested this correctly after all the refactors...
Basically the issue is that the websocket connection would only come
online when the API state changed to full `Online`. In particular
the connector would not attempt to connect when the API state was
`RequiresSecondFactorAuth`, giving the link-based flow no chance to
actually work.
The change in `WebSocketNotificationsClientConnector` is relevant in
that queueing requests does nothing before the API state changes to full
`Online`. It also cleans up things a bit code-wise so... win?
And yes, this means that the _other_ `PersistentEndpointClientConnector`
implementations (i.e. SignalR connectors) will also come online earlier
after this. Based on previous discussions
(https://github.com/ppy/osu/pull/25480#discussion_r1395566545) I think
this is fine, but if it is _not_ fine, then it can be fixed by exposing
a virtual that lets a connector to decide when to come alive, I guess.
