After way too much time investigating this, the encoding situation is
not great right now.
- Stable sets the "default code page" to be used for encoding filenames
to Shift-JIS (932):
c29ebd7fc5/osu!/GameBase.cs#L3099
- Lazer does nothing (therefore using UTF-8).
When importing to lazer, stable files are assumed to be UTF-8. This
means that the linked beatmaps don't work correctly. Forcing lazer to
decompress *and* compress using Shift-JIS will fix this.
Here's a rough idea of how things look for japanese character filenames
in current `master`:
| | stable | lazer |
|--------|--------|--------|
| export encoding | shift-jis | utf8 |
| utf8 [bit flag](https://superuser.com/a/1507988) set | ❌ | ❌ |
| import stable export osz | ✅ | ❌ |
| import lazer export osz | ❌ | ✅ |
| windows unzip | ❌ | ❌ |
| macos unzip | ✅ | ✅ |
and after this change
| | stable | lazer |
|--------|--------|--------|
| export encoding | shift-jis | shift-jis |
| utf8 [bit flag](https://superuser.com/a/1507988) set | ❌ | ❌ |
| import stable export osz | ✅ | ✅ |
| import lazer export osz | ✅ | ✅ |
| windows unzip | ❌ | ❌ |
| macos unzip | ✅ | ✅ |
A future endeavour to improve compatibility would be to look at setting
the utf8 flag in lazer, switching the default to utf8, and ensuring the
stable supports this flag (I don't believe it does right now).
As it turns out, the "lower" and "upper" estimates of the combo portion
of the score being converted were misnomers. In selected cases
(scores with high accuracy but combo being lower than max by more than
a few objects) the janky score-based math could overestimate the count
of remaining objects in a map. For instance, in one case the numbers
worked out something like this:
- Accuracy: practically 100%
- Max combo on beatmap: 571x
- Max combo for score: 551x
The score-based estimation attempts to extract a "remaining object
count" from score, by doing something along of sqrt(571^2 - 551^2). That
comes out to _almost 150_. Which leads to the estimation overshooting
the total max combo count on the beatmap by some hundred objects.
To curtail this nonsense, enforce some basic invariants:
- Neither estimate is allowed to exceed maximum achievable
- Ensure that lower estimate is really lower and upper is really upper
by just looking at the values and making sure that is so rather than
just saying that it is.
This subtle detail was messing with server-side score import flows.
Server-side, legacy total score will *already* be in `LegacyTotalScore`
from the start, and `TotalScore` will be zero until recomputed via
`StandardisedScoreMigrationTools.UpdateFromLegacy()` - so in that
context, attempting to move it across is incorrect.
Partially addresses https://github.com/ppy/osu/discussions/26416
As pointed out in the discussion thread above, the total score
conversion process for mania was using accuracy directly from the
replay. In mania accuracy is calculated differently in score V1 than in
score V2, which meant that scores coming from stable were treated more
favourably (due to weighting GREAT and PERFECT equally).
To fix, recompute accuracy locally and use that for the accuracy
portion.
Note that this will still not be (and cannot be made) 100% accurate, as
in stable score V2, as well as in lazer, hold notes are *two*
judgements, not one as in stable score V1, meaning that full and correct
score statistics are not available without playing back the replay.
The effects of the change can be previewed on the following spreadsheet:
https://docs.google.com/spreadsheets/d/1wxD4UwLjwcr7n9y5Yq7EN0lgiLBN93kpd4gBnAlG-E0/edit#gid=1711190356
Top 5 changed scores with replays:
| score | master | this PR | replay |
| :------------------------------------------------------------------------------------------------------------------------------- | ------: | ------: | ------: |
| [Outlasted on Uwa!! So Holiday by toby fox [[4K] easy] (0.71\*)](https://osu.ppy.sh/scores/mania/460404716) | 935,917 | 927,269 | 920,579 |
| [ag0 on Emotional Uplifting Orchestral by bradbreeck [[4K] Rocket's Normal] (0.76\*)](https://osu.ppy.sh/scores/mania/453133066) | 921,636 | 913,535 | 875,549 |
| [rlarkgus on Zen Zen Zense by Gom (HoneyWorks) [[5K] Normal] (1.68\*)](https://osu.ppy.sh/scores/mania/458368312) | 934,340 | 926,787 | 918,855 |
| [YuJJun on Harumachi Clover by R3 Music Box [4K Catastrophe] (1.80\*)](https://osu.ppy.sh/scores/mania/548215786) | 918,606 | 911,111 | 885,454 |
| [Fritte on 45-byou by respon feat. Hatsune Miku & Megpoid [[5K] Normal] (1.52\*)](https://osu.ppy.sh/scores/mania/516079410) | 900,024 | 892,569 | 907,456 |
Standardised score conversion would return a negative total score for
https://osu.ppy.sh/scores/taiko/182230346.
The underlying reason for this is that the estimation of the accuracy
portion for a score can be above the actual accuracy portion in the
taiko ruleset.
When calculating the maximum accuracy portion achievable,
`TaikoLegacyScoreSimulator` will include the extra 300 points from
a double hit on a strong hit, per strong hit. However, this double hit
is not factored into accuracy.
Both of the aforementioned facts mean that in taiko
maximumLegacyAccuracyScore * score.Accuracy
- which normally in other rulesets can be used pretty reliably as the
exact number of points gained from the accuracy portion - is an
estimate in the case of taiko, and an _upper_ estimate at that,
because it implicitly assumes that the user has also hit
`score.Accuracy` percent of all double hits possible in the beatmap. If
this assumption is not upheld, then the user will have earned _less_
points than that from the accuracy portion, which means that the combo
proportion estimate will go below zero.
It is possible that this has happened on other scores before, but did
not result in the total score going negative as the accuracy portion
gained would have counteracted the effect of that due to being larger in
magnitude than the score loss incurred from the negative combo
portion. In the case of the score in question this was not the case due
to very low accuracy _and_ very low max combo.
Co-authored-by: Zyf <zyfarok@gmail.com>
Closes https://github.com/ppy/osu/issues/25860
Users reported that some stable scores would convert to large negative
total scores in lazer after the introduction of combo exponent. Those
large negative total scores were actually mangled NaNs.
The root cause of this was the following calculation going below zero
unexpectedly:
8e8d9b2cd9/osu.Game/Database/StandardisedScoreMigrationTools.cs (L323)
which then propagates negative numbers onward until
8e8d9b2cd9/osu.Game/Database/StandardisedScoreMigrationTools.cs (L337)
which yields a NaN due to attempting to take the square root of a
negative number.
To fix, clamp `comboPortionInScoreV1` to sane limits: to
`comboPortionFromLongestComboInScoreV1` from below, and to
`maximumAchievableComboPortionInScoreV1` from above. This is a less
direct fix than perhaps imagined, but it seems like a better one as it
will also affect the calculation of both the lower and the upper
estimate of the score.
