1682 Commits

• Fix external edit filename sanitising unintentionally dropping folder separators (#34945)

  * Add failing test coverage showing external edits not working with folders
  
  * Fix external edit filename sanitising unintentionally dropping folder separators
  
  Closes https://github.com/ppy/osu/issues/34929.

  Dean Herbert · 2025-09-07 21:47:20 +09:00
  645518f5bd
• Fix external edit operations failing due to invalid filenames
  Bartłomiej Dach · 2025-09-03 09:26:06 +02:00
  51e7593446
• Fix copy-paste fail in log message
  Bartłomiej Dach · 2025-08-25 13:43:03 +02:00
  a2bf8e3988
• Fix submission & rank date failing every launch for some users

  Addresses https://github.com/ppy/osu/discussions/34705, I suppose.
  
  The cagey tone of that statement is because this change merely papers
  over the issue. The issue in question for the user that reported this is
  that they have a bunch of very old beatmaps, whose md5 hashes do not
  match the online hashes, that need updating. The submission/rank date
  population was running every single time for these, and failing every
  time, because there is really not much useful that the lookup *can* do.
  
  Because mappers have made `OnlineID` essentially useless for determining
  the provenance of a beatmap due to reusing them to "fix" beatmap
  submission failures, online IDs have been explicitly disallowed from use
  in any sort of beatmap lookup flow. The only things that are allowed to
  be used are: md5 of the beatmap, and filename as a fallback for very old
  beatmaps / beatmap packs.
  
  If the user has local beatmaps with md5 not matching online, chances are
  that any metadata lookups are likely to fail or return bogus data. At
  that point my personal feeling is that backpopulation flows should leave
  such beatmaps well alone and the user should just go update the beatmap
  themselves.
  
  I am aware that updating 124 individual beatmap sets would - in the
  current state of things - would probably be a ridiculously onerous thing
  to do, and that people have been asking multiple times for a facility to
  update all local beatmaps at once, but that discussion is out of scope
  at this stage.

  Bartłomiej Dach · 2025-08-20 09:21:35 +02:00
  c894969d17
• Reduce number of realm subscription triggers from backpopulation of user tags

  Probably partially alleviates https://github.com/ppy/osu/issues/34580.

  Bartłomiej Dach · 2025-08-12 09:53:17 +02:00
  4439bff822
• Fix metadata cache version queries failing hard if the image is corrupted

  This was not fatal but with this change it should recover in a slightly
  better way.
  
  Also incidentally fixes the cache refetch potentially being attempted
  twice by each of the two background population tasks that use it.

  Bartłomiej Dach · 2025-08-07 11:10:02 +02:00
  dac643121a
• Add support for grouping beatmaps by collections

  Takes changes in https://github.com/ppy/osu/pull/34233 and removes *all*
  of the complexity.
  
  Supersedes and closes #34233.
  
  Contains same caveat that we can only display one beatmap once in the
  carousel; this will be addressed separately.

  Dean Herbert · 2025-08-01 16:23:26 +09:00
  c3396a93a6
• Update inline comment
  Bartłomiej Dach · 2025-07-30 14:27:35 +02:00
  d4794dbc9a
• Apply code review suggestion
  Bartłomiej Dach · 2025-07-30 14:20:13 +02:00
  d49c924709
• Only perform user tag backpopulation after the local metadata cache updates
  Bartłomiej Dach · 2025-07-29 11:50:41 +02:00
  8307947a10
• Add back-population operation for user tags
  Bartłomiej Dach · 2025-07-23 10:35:32 +02:00
  1709811458
• Fix comment

  Co-authored-by: Dean Herbert <pe@ppy.sh>

  Bartłomiej Dach · 2025-07-22 09:44:35 +02:00
  5e0219c58f
• Merge branch 'master' into store-pause-count
  Bartłomiej Dach · 2025-07-22 08:44:16 +02:00
  b08ba3c9e4
• Add initial support for filtering by user tags in song select

  The way that this works is that it plugs into the online request to
  retrieve the beatmap set that the client is already performing, and
  stores user tag data to the local realm database.
  
  This means that for now user tags will only populate for beatmaps that
  the user has displayed on song select which is obviously subpar. I plan
  to follow this change up by adding user tag state dumps to `online.db`
  and using that data for initial tag population to make the majority case
  (ranked beatmaps) work.
  
  Note that several decisions were made here that are potential discussion
  points:
  
  - `RealmPopulatingOnlineLookupSource` is set up such that it can be the
    middle man / redirection point for similar flows that we need and we
    are currently missing, such as storing guest difficulty information,
    or storing the user's current best score on a beatmap (handy for rank
    achieved sorting / filtering / etc.)
  
  - The user tags are stored in `BeatmapMetadata` which breaks the
    longstanding assumption that you can arbitrarily pull out a metadata
    instance from any of the beatmaps in a set and get essentially the
    same object back.
  
    I've attempted to constrain this some by not adding user tags to
    the `IBeatmapMetadataInfo` interface through which `BeatmapSetInfo`
    exposes metadata further, but I warn in advance that this is
    a temporary state of affairs and I will make it worse in the future
    when `BeatmapMetadata.Author` becomes `Authors` plural in order to
    support guest mapper display (and direct guest difficulty submission).
  
  - The syntax for searching via user tags is chosen to mostly match web -
    it's `tag=`, with support for all of the string matching modes song
    select already has (bare word for substring, `""` quotes for phrase
    isolated by whitespace, `""!` for exact full match).

  Bartłomiej Dach · 2025-07-15 11:09:40 +02:00
  8cb81974eb
• Store score pause count to realm database
  Bartłomiej Dach · 2025-07-07 11:00:31 +02:00
  c83dcdc915
• Fix skins containing subdirectories breaking on external edit on windows

  Closes https://github.com/ppy/osu/issues/33994.
  
  The reason for the breakage is that `Directory.EnumerateFiles()` used in
  
  	https://github.com/ppy/osu/blob/b1435d35e56eed08a57d1909fa0b16e67bd9c2a2/osu.Game/Skinning/SkinImporter.cs#L63
  
  will use the primary platform directory separator character, which is
  `\` on windows and `/` on unices. The internal realm storage structure
  is expecting paths to be normalised to the unix convention, which is
  evident in
  
  	https://github.com/ppy/osu/blob/b1435d35e56eed08a57d1909fa0b16e67bd9c2a2/osu.Game/Database/RealmArchiveModelImporter.cs#L499
  
  on the write side and in
  
  	https://github.com/ppy/osu/blob/b1435d35e56eed08a57d1909fa0b16e67bd9c2a2/osu.Game/Skinning/RealmBackedResourceStore.cs#L50
  
  on the read side.
  
  Rather than applying this locally to the skin importer I kinda think
  it's better to have this call in `ModelManager` to hopefully avoid
  future footgunnage of this kind.

  Bartłomiej Dach · 2025-07-03 08:16:51 +02:00
  71210bedeb
• Fix dodgy threading

  The fact that the stuff "just worked" previously due to one load-bearing
  detach in a random location is really scary because a lot of this was
  just not written the way it is supposed to be.

  Bartłomiej Dach · 2025-06-30 14:17:27 +02:00
  2e0e7ff3c2
• Merge branch 'master' into skin-mounting
  Bartłomiej Dach · 2025-06-30 12:53:55 +02:00
  368850967b
• Fix stable scores importing with a LegacyOnlineID of 0

  Closes https://github.com/ppy/osu/issues/33435.
  
  The root cause of the issue is that the user's database contained a
  whole lot of scores with `LegacyOnlineID` of 0, which would trip up
  
  	https://github.com/ppy/osu/blob/97e6187f0d7c3dbee896596a623e34627135bf0e/osu.Game/Extensions/ModelExtensions.cs#L128-L129
  
  as that method would thus consider scores that are not the same as the
  same because of the zero, which later trips up
  
  	https://github.com/ppy/osu/blob/97e6187f0d7c3dbee896596a623e34627135bf0e/osu.Game/Screens/Ranking/SoloResultsScreen.cs#L79
  
  which ends up inserting `Score` into the list several times, which
  causes the crash.
  
  You might remember that I tried to fix this once before in
  https://github.com/ppy/osu/pull/24794. What I did not realise, however,
  is that stable *can still produce replays* that have an online ID of
  zero in them, because zero *is just `default(long)`*:
  
  	https://github.com/peppy/osu-stable-reference/blob/7205341bb70000a87fa1bd54e7642772e2af85d7/osu!/GameplayElements/Scoring/Score.cs#L123
  	https://github.com/peppy/osu-stable-reference/blob/7205341bb70000a87fa1bd54e7642772e2af85d7/osu!/GameplayElements/Scoring/Score.cs#L350
  
  The alternative way of fixing this would be just to change
  `MatchesOnlineID` to reject zeroes, but I think this is a saner overall
  direction.

  Bartłomiej Dach · 2025-06-10 11:54:11 +02:00
  d4fc6693b4
• Use UTF-8 encoding when exporting skin archives

  Closes https://github.com/ppy/osu/issues/33337.
  
  See also: https://github.com/ppy/osu/pull/28034#issuecomment-2084450285

  Bartłomiej Dach · 2025-06-03 13:54:54 +02:00
  fe325ba861
• Remove redundant public specfications in interfaces

  Rider EAP being noisy about this now (smoogi will be happy :D).

  Dean Herbert · 2025-05-26 17:14:41 +09:00
  cb0592d711
• Add debounce to difficulty calculation operations in churn scenarios
  Dean Herbert · 2025-05-19 16:16:33 +09:00
  192a1642d4
• Fix presenting score potentially dying due to deleted beatmap

  - Closes https://github.com/ppy/osu/issues/27168
  - Closes https://github.com/ppy/osu/issues/32930
  
  It's a little manual (if you perform any of the scenarios in the issues
  above on this branch, the first click will re-import the beatmap but not
  start the replay, and only the second will play it), but maybe fine?

  Bartłomiej Dach · 2025-04-24 08:47:08 +02:00
  c52dce0f38
• Fix thread safety when calling BeatmapStore.GetBeatmapSets

  While usually we'd handle this locally by moving bind operations to
  `LoadComponent`, this component was explicitly made to be used in
  asynchronous scenarios (to allow cases like song select to coexist with
  realm without adding huge compliexities to the classes locally).
  
  So I think it makes sense to hide this as an implementation detail. The
  locked segments should all be quite fast to run so I do not see a
  performance issue with lock contention here.

  Dean Herbert · 2025-04-16 15:53:20 +09:00
  e8af0dabea
• Fix timeout during diffcalc causing batch import to bail

  Closes https://github.com/ppy/osu/discussions/32628.

  Dean Herbert · 2025-04-03 14:30:47 +09:00
  21c6756336
• Re-fetch status of any beatmaps stuck in qualified status

  Best we do this rather than leaving it up to users to fix their broken
  beatmaps.
  
  https://github.com/ppy/osu/discussions/32406
  https://github.com/ppy/osu/discussions/32431

  Dean Herbert · 2025-03-19 14:35:43 +09:00
  76f286a01b
• Ensure realm database file is touched on startup

  Closes https://github.com/ppy/osu/discussions/32304.

  Dean Herbert · 2025-03-11 15:47:51 +09:00
  75bd101c9e
• Pull actual diffcalc out of realm transaction
  Bartłomiej Dach · 2025-03-10 10:36:19 +01:00
  6b1472b070
• Cache ruleset instance to avoid instantiation per beatmap processed
  Dean Herbert · 2025-03-10 17:17:45 +09:00
  7ca9d8392d
• Ensure that star rating reprocessing does not incur online lookup requests

  Yesterday after the lazer release there was a bit of a spike in the
  number of osu-web requests pointed at `/api/v2/beatmaps/lookup`
  specifically. The most likely reason for this is that prior to this
  commit, the star rating recalculation was fully performed by the
  `BeatmapUpdater.Process()` flow.
  
  This process does full metadata lookups, and while it *will* attempt
  to use the local `online.db` metadata cache, it *will* also fall back to
  API requests if the local metadata fetch fails. While that means that
  the local cache likely saved us from a doomsday scenario here, it *also*
  is the case that all of that metadata lookup stuff is *entirely
  unnecessary* when wanting to just update star ratings.
  
  Therefore, this splits out only the part relevant to star ratings
  as a separate background process, so that it can run completely
  locally.

  Bartłomiej Dach · 2025-03-07 10:48:29 +01:00
  12fa96de25
• Implement special exporter intended specifically for submission flows
  Bartłomiej Dach · 2025-02-05 14:01:37 +01:00
  fff99a8b40
• Do not double up first control point of path
  Bartłomiej Dach · 2025-01-31 09:01:42 +01:00
  20280cd195
• Move control point double-up logic to LegacyBeatmapExporter

  Done for two reasons:
  
  - During review it was requested for the logic to be moved out of
    `BezierConverter` as `BezierConverter` was intended to produce
    "lazer style" sliders with per-control-point curve types,
    as a future usability / code layering concern.
  
  - It is also relevant for encode-decode stability. With how the logic
    was structured between the Bezier converter and the legacy beatmap
    encoder, the encoder would leave behind per-control-point Bezier curve
    specs that stable ignored, but subsequent encodes and decodes in lazer
    would end up multiplying the doubled-up control points ad nauseam.
    Instead, it is sufficient to only specify the curve type for the
    head control point as Bezier, not specify any further curve types
    later on, and instead just keep the double-up-control-point for new
    implicit segment logic which is enough to make stable cooperate
    (and also as close to outputting the slider exactly as stable would
    have produced it as we've ever been)

  Bartłomiej Dach · 2025-01-30 15:28:47 +01:00
  b4f63da048
• Merge pull request #31509 from bdach/local-metadata-cache-corruption

  Refetch local metadata cache if corruption is detected

  Dean Herbert · 2025-01-23 02:43:14 +09:00
  bf23c67603
• Merge pull request #31604 from peppy/fix-song-select-right-click-handling

  Fix song select right click handling

  Bartłomiej Dach · 2025-01-22 10:00:03 +01:00
  2623d8dcac
• Remove outdated ef related catch-when usage
  Dean Herbert · 2025-01-22 17:04:30 +09:00
  26ef23c9a9
• Switch IsImportant to init property isntead of virtual
  Dean Herbert · 2025-01-22 16:41:26 +09:00
  4cf4b8c73d
• Remove right click default for absolute scroll
  Dean Herbert · 2025-01-21 17:24:41 +09:00
  66be9f2d1b
• Update realm xmldoc references
  Dean Herbert · 2025-01-08 01:35:56 +09:00
  125d652dd8
• Merge branch 'master' into upgrade-packages
  Dean Herbert · 2025-01-08 01:23:28 +09:00
  18ff25bc6d
• Round object coordinates to nearest integers rather than truncating

  Addresses https://github.com/ppy/osu/issues/31256.

  Bartłomiej Dach · 2024-12-27 11:10:29 +01:00
  e9762422b3
• Add migration
  Dean Herbert · 2024-12-26 15:46:12 +09:00
  2a374c0695
• Add explanatory comment for schema version bump
  Bartłomiej Dach · 2024-12-24 08:21:27 +01:00
  d9be172647
• Add migration to update users which have previous default bindings for beat snap
  Dean Herbert · 2024-12-23 22:14:03 +09:00
  9ff4a58fa3
• Merge branch 'master' into skin-mounting
  Bartłomiej Dach · 2024-12-12 12:16:22 +09:00
  09d55ed306
• Merge pull request #31076 from peppy/beatmap-store-interface

  Access beatmap store via abstract base class

  Bartłomiej Dach · 2024-12-11 17:48:32 +09:00
  4e4a99decc
• Remove BeatmapCarousel testing backdoor
  Dean Herbert · 2024-12-11 16:28:51 +09:00
  a868c33380
• Access beatmap store via abstract base class

  The intention here is to make things more testable going forward.
  Specifically, to remove the "back-door" entrance into `BeatmapCarousel`
  where `BeatmapSets` can be set by tests and bypas/block realm retrieval.

  Dean Herbert · 2024-12-11 16:28:51 +09:00
  c94b393e30
• Revert unnecessary complexity
  Salman Alshamrani · 2024-12-11 01:03:55 -05:00
  e0aec6f907
• Use temporary storage for exported files on iOS
  Salman Alshamrani · 2024-12-09 08:16:37 -05:00
  0c0dcb1e15