h1or4dme8/osu-lazer
1
0
Fork 0
mirror of https://github.com/ppy/osu.git synced 2024-11-14 16:37:26 +08:00
Code Issues Releases Wiki Activity
67,572 Commits 3 Branches 654 Tags 362 MiB
f88f05717a
Commit Graph

1577 Commits

Author SHA1 Message Date
Dean Herbert
6ad86ce5b7 Run collection import process asynchronously 
Actually required to avoid deadlocking..
 2022-07-31 01:06:57 +09:00
Dean Herbert
80ffa2cf20 Move collection database rather than deleting post-migration for safety 2022-07-31 00:54:00 +09:00
Dean Herbert
8e06d55960 Fix collection migration incorrectly running asynchronously 2022-07-31 00:53:39 +09:00
Dean Herbert
cb52b696ed
Merge branch 'master' into bypass-local-metadata-cache 2022-07-29 16:05:54 +09:00
Dean Herbert
c35da62224 Add flow for bypassing local cache lookups when refreshing beatmap metadata 2022-07-28 16:33:51 +09:00
Dean Herbert
2209afd0e8 Mark Live methods as InstantHandleAttribute 2022-07-28 15:03:08 +09:00
Dean Herbert
1669208a54 Add migration of existing collections database 2022-07-28 13:57:21 +09:00
Dean Herbert
41393616d8 Replace BeatmapCollection with RealmBeatmapCollection 2022-07-27 19:35:09 +09:00
Dean Herbert
9c543fef48 Remove CollectionManager 2022-07-27 18:45:33 +09:00
Dean Herbert
6b73f7c7ec Split out legacy import path from realm manager 2022-07-27 15:04:09 +09:00
Dean Herbert
9173271900 Fix new update pathway not actually being used 2022-07-26 17:58:09 +09:00
Dean Herbert
8370ca9765 Add ImportAsUpdate method to IModelImporter to avoid otehr changes 2022-07-26 15:49:04 +09:00
Dean Herbert
8a0c8f5fd8 Fix some realm pieces not being cleaned up 2022-07-25 19:51:19 +09:00
Dean Herbert
2e14d8730c Move implementation of updating a beatmap to BeatmapImporter 2022-07-25 19:51:19 +09:00
Dean Herbert
b7f6413bce Fix old version of beatmap potentially not being deleted during update flow 
This can happen if the online IDs are not present in the `.osu` files.
Previously this was only working due to the early logic in the import
process (that relies on matching all online IDs perfectly).
 2022-07-25 19:51:19 +09:00
Dean Herbert
94cd641fb4 Change migration to trigger reprocessing on every local beatmap 
Was originally relying on the fact that this would be triggered due to a
null `LastOnlineUpdate`, but wouldn't cover the case of beatmaps with no
`OnlineID`.
 2022-07-21 18:31:12 +09:00
Dean Herbert
d5e0dba9da Change default value of StarRating to -1 2022-07-21 18:20:46 +09:00
Dean Herbert
68f28ff660 Add last applied version to RulesetInfo 2022-07-21 18:15:25 +09:00
Dan Balasescu
2b399ec7ad
Merge branch 'master' into date-submitted-ranked 2022-07-20 18:36:57 +09:00
Dean Herbert
6357223341 Fix incorrect DI fetch and apply nullability to ModelDownloader 2022-07-19 20:06:34 +09:00
Dean Herbert
30daa0fd44 Add ranked and submitted date storage and filtering 2022-07-19 19:55:56 +09:00
Dean Herbert
17046b0553 Add basic handling of download failures 2022-07-19 19:01:23 +09:00
Dean Herbert
6ea380d649 Add new properties to BeatmapInfo to track online hash and updates 2022-07-19 17:57:01 +09:00
Dean Herbert
51f91fe62e Update naming 2022-07-18 16:17:20 +09:00
Salman Ahmed
4e7156cee8 Store user country on databased scores 2022-07-16 06:39:05 +03:00
Dean Herbert
c8c79d2185 Standardise HasReplay implementation (and remove from persisting to realm) 2022-07-15 16:14:21 +09:00
Dean Herbert
8820ea4006 Add last played date to BeatmapInfo 2022-07-13 16:36:43 +09:00
Dean Herbert
8b6665cb5b Ensure initial beatmap processing is done inside the import transaction 2022-07-11 02:51:54 +09:00
Dan Balasescu
0bc332e90b
Merge pull request #19029 from peppy/fix-skin-import-duplicate 
Fix skins potentially being duplicated on batch import
 2022-07-08 14:18:14 +09:00
Dean Herbert
c53dd4a703 Fix editor saving not updating BeatmapSetInfo's hash 2022-07-08 02:33:14 +09:00
Dean Herbert
cf1da1dd18 Fix skins potentially being duplicated on batch import 
Resolves https://github.com/ppy/osu/discussions/19024#discussioncomment-3099200
 2022-07-07 22:39:07 +09:00
Dean Herbert
ac216d94a8 Fix transaction not being disposed 2022-07-07 18:15:38 +09:00
Dean Herbert
5197d0fa9e Add automatic transaction handling to realm helper methods 2022-07-07 17:36:25 +09:00
Dean Herbert
a1b6ec60c8 Add statistics display for MemoryCachingComponents 
Never sure if these are working as they should (or how well they are
working). This helps quite a bit.
 2022-07-06 18:38:40 +09:00
Dean Herbert
33db508301 Add note regarding why the realmRetrievalLock return is done on the async thread 2022-07-05 16:23:10 +09:00
Dean Herbert
5adec2c738 Ensure blocking restoration only completes after update callback work is completed 2022-07-05 14:39:14 +09:00
Dean Herbert
1b98936328 Rename realm ThreadLocal to better convey what it's doing 
Every time I looked at this code I have to re-learn what it's doing.
Changing these variable names should help quite a bit.
 2022-07-05 13:38:25 +09:00
Dean Herbert
506409a9c4 Fix realm backup creation failing when run from RealmAccess constructor 
At the point of construction, we are not on the update thread, but it
doesn't really matter at this point because there's no other usages.
 2022-07-04 16:45:23 +09:00
Salman Ahmed
e28ee8bc7a Count time spent upwards to display attempts correctly 2022-07-02 10:46:52 +03:00
Dean Herbert
93809a92d4 Fix clashing error messaging during realm block operations 2022-07-02 16:30:04 +09:00
Dean Herbert
4fd47b5fa0 Add more verbose logging to realm blocking process 2022-07-02 12:36:56 +09:00
Dan Balasescu
3b1842a2c2
Merge pull request #18835 from peppy/beatmap-update-flow 
Split out beatmap update tasks to `BeatmapUpdater` and invoke from editor save flow
 2022-07-01 20:28:04 +09:00
Dean Herbert
447b496eff
Merge branch 'master' into realm-fix-async-write-after-disposal 2022-06-30 19:07:15 +09:00
Dean Herbert
2a73210865 Add xmldoc and update parameter naming for MemoryCachingComponent.Invalidate flow 2022-06-30 17:17:06 +09:00
Dean Herbert
98938821e5 Merge branch 'master' into beatmap-update-flow 2022-06-30 16:44:17 +09:00
Dean Herbert
78d86fd3ff Fix PerformWrite not rolling back transaction on exception 2022-06-30 15:15:09 +09:00
Dean Herbert
7cb4e32c17 Add one more lock to appease CI 2022-06-29 22:45:19 +09:00
Dean Herbert
a3b4a515fc Merge branch 'master' into realm-fix-async-write-after-disposal 2022-06-29 22:39:05 +09:00
Dean Herbert
32af4e41ea Add back thread safety and locking as required 2022-06-29 20:56:01 +09:00
Dean Herbert
ecdb30d215 Fix one more case of collection modification during enumeration 
https://sentry.ppy.sh/share/issue/a61c27b2a63a4a6aa80e75873f9d87ca/
 2022-06-29 02:28:23 +09:00
Dean Herbert
e10ac45fd7 Remove probably redundant realmLock 
As far as I can tell all accesses are safe due to update thread
guarantees. The only weird one may be async writes during a
`BlockAllOperations`, but the `Compact` loop should handle this quite
amicably.
 2022-06-28 16:55:54 +09:00
Dan Balasescu
396e7fc166
Merge pull request #18862 from peppy/empty-beatmap-file-fix-2 
Fix second case of empty beatmaps being reported to sentry as errors
 2022-06-27 20:37:57 +09:00
Dean Herbert
f355c3a3c9
Merge pull request #18864 from peppy/fix-realm-audio-track-test-fail 
Centralise and harden editor-ready-for-use check
 2022-06-27 19:42:49 +09:00
Dean Herbert
83982d258d Throw immediately if attempting to WriteAsync after disposed 2022-06-27 19:34:28 +09:00
Dean Herbert
c39c99bd43 Ensure all async writes are completed before realm is disposed 2022-06-27 19:21:05 +09:00
Dean Herbert
f6a61472c4 Fix occasional failure in realm test proceedings due to incorrect Debug.Assert 
After a `BlockAllOperations`, the restoration of the `updateRealm`
instance is not instance. It is posted to a `SynchronizationContext`.
The assertion which has been removed in this commit was assuming it
would always be an immediate operation.

To ensure this works as expected, I've tracked the initialised state via
a new `bool`.

```csharp
System.TimeoutException : Attempting to block for migration took too long.

  1) Host threw exception System.AggregateException: One or more errors occurred. (: )
 ---> NUnit.Framework.AssertionException: :
   at osu.Framework.Logging.ThrowingTraceListener.Fail(String message1, String message2)
   at System.Diagnostics.TraceInternal.Fail(String message, String detailMessage)
   at System.Diagnostics.TraceInternal.TraceProvider.Fail(String message, String detailMessage)
   at System.Diagnostics.Debug.Fail(String message, String detailMessage)
   at osu.Game.Database.RealmAccess.BlockAllOperations() in /opt/buildagent/work/ecd860037212ac52/osu.Game/Database/RealmAccess.cs:line 813
   at osu.Game.OsuGameBase.<>c__DisplayClass108_1.<Migrate>b__0() in /opt/buildagent/work/ecd860037212ac52/osu.Game/OsuGameBase.cs:line 449
   at osu.Framework.Threading.ScheduledDelegate.RunTaskInternal()
   at osu.Framework.Threading.Scheduler.Update()
   at osu.Framework.Graphics.Drawable.UpdateSubTree()
   at osu.Framework.Graphics.Containers.CompositeDrawable.UpdateSubTree()
```

https://teamcity.ppy.sh/buildConfiguration/Osu_Build/322?hideProblemsFromDependencies=false&hideTestsFromDependencies=false&expandBuildTestsSection=true
 2022-06-27 18:01:12 +09:00
Dean Herbert
01487a1185 Add assertion on realm re-fetch being not-null 2022-06-27 15:52:45 +09:00
Dean Herbert
da61d0547f Include archive name in import log output 2022-06-27 15:24:25 +09:00
Dean Herbert
31a447fda0 Update parameter discards 2022-06-24 21:26:19 +09:00
Dean Herbert
30b3973c9f Difficulty cache invalidation flow 2022-06-24 21:02:38 +09:00
Bartłomiej Dach
26c5b59f6d
Replace usages of string.To{Lower,Upper}() 2022-06-24 11:57:45 +02:00
Dean Herbert
51268d0cc8 Throw on Wait failure in a few remaining cases 2022-06-23 15:28:21 +09:00
Dean Herbert
7ef8b7df5f Add timeout for all ManualResetEventSlim.Wait invocations 
Timeout values were taken as best-guesses of upper values we'd expect
from sane execution.
 2022-06-23 14:46:00 +09:00
Dean Herbert
948c28f415 Fix collection modified during BlockAllOperations if any subscriptions have been established 2022-06-22 20:34:05 +09:00
Dan Balasescu
816fd338cb
Fix typo 2022-06-21 19:57:27 +09:00
Dean Herbert
f74b4ac277 Fix blocking overhead when calling WriteAsync 2022-06-21 17:15:25 +09:00
Dean Herbert
ba394f2831 Remove IPostsImports interface (and move to IModelImporter) 2022-06-20 18:53:48 +09:00
Dean Herbert
e732c5a2d7 Add PostImport method matching PreImport 2022-06-20 18:40:23 +09:00
Dean Herbert
45c5013f09 Remove default value of RealmArchiveModelImporter.HandledExtensions 
Not used anywhere and probably not wanted ever.
 2022-06-20 18:40:23 +09:00
Dean Herbert
dbae4c6f5a PostImport -> PresentImport 2022-06-20 18:36:29 +09:00
Dean Herbert
4aa32038c3 Add note about stream provided to ImportTask being disposed implicitly 2022-06-20 15:38:44 +09:00
Dean Herbert
882e1c69e5 Move nested task back outside try-catch to avoid double-error 2022-06-20 15:36:44 +09:00
Dean Herbert
678b18dde6 Ensure any non-MemoryStream streams are closed as part of ImportTask 2022-06-20 15:29:56 +09:00
Dean Herbert
dd93c7359e Update xmldoc for importFromArchive method 2022-06-20 15:25:43 +09:00
Dean Herbert
92f1a2958c Rename Import(TModel) to ImportModel to differentiate from other import methods 2022-06-20 15:18:07 +09:00
Dean Herbert
a6c8a832aa Remove Import(ArchiveReader) and redirect existing usages to Import(ImportTask) 2022-06-20 15:14:57 +09:00
Dan Balasescu
f8830c6850 Automated #nullable processing 2022-06-17 16:37:17 +09:00
Bartłomiej Dach
651862fee0
Merge branch 'master' into database-migration-reliability 2022-06-16 17:19:58 +02:00
Dean Herbert
bf6c6682bc Move null to finally 2022-06-16 23:37:24 +09:00
Dean Herbert
7809566f16 Add explanatory comments 2022-06-16 23:31:49 +09:00
Dean Herbert
7b0fad6461 Null disposal token after use 2022-06-16 23:29:33 +09:00
Dean Herbert
448eee051a Update filename to match new class name 2022-06-16 19:41:14 +09:00
Dean Herbert
ce3d3a967c Reimplement missing methods 2022-06-16 19:05:25 +09:00
Dean Herbert
72c5b9009d Remove local realm fields in manager classes 2022-06-16 18:56:53 +09:00
Dean Herbert
f3984d98e6 Remove RealmArchiveModelManager from hierarchy 2022-06-16 18:53:13 +09:00
Dean Herbert
1f3e1b2d97 Combine BeatmapManager and BeatmapModelManager into one class 2022-06-16 18:07:04 +09:00
Dean Herbert
4526f8c07d Move database backup creation to async thread where possible 2022-06-16 17:01:17 +09:00
Dean Herbert
739a696467 Ensure reading of existing databases when making backups doesn't create a file 2022-06-16 16:48:06 +09:00
Dean Herbert
f23ddfe6cc Move remaining realm classes out of Stores namespace 2022-06-15 21:56:00 +09:00
Dean Herbert
cca7633e59 Rename variables and remove unnecessary nested call to GetFullPath 2022-06-15 15:05:51 +09:00
Salman Ahmed
22c09ec893 Handle subdirectories during beatmap stable import 2022-06-15 07:33:21 +03:00
Dean Herbert
e96c5ce703 Move and correct xmldoc to interfaces 2022-06-14 20:00:16 +09:00
Dean Herbert
d73c0ca659 Remove unused methods from IModelImporter 
All the xmldoc is already present in `RealmArchiveModelImporter` so
nothing is lost here.
 2022-06-14 19:42:11 +09:00
Dean Herbert
1b4c89c418 Update realm to latest version 
Contains minor changes to async usage in line with upstream API changes.

I believe a feedback issue we were seeing with offset changes (the only
component using async write flow) may have been resolved by these
upstream changes (see [release
notes](https://github.com/realm/realm-dotnet/releases/tag/10.14.0)) but
am not investigating further just yet.
 2022-06-03 14:16:10 +09:00
Dean Herbert
45841673f6 Update OnlineLookupCache to use async version of Perform call 2022-05-30 16:13:13 +09:00
Dean Herbert
4633eed78c Merge branch 'master' into localisable-filter-items 2022-05-20 23:55:16 +09:00
Dean Herbert
70bd40ce44 Fix incorrect count of beatmaps available to import 2022-05-19 14:01:24 +09:00
Dean Herbert
1b7ec1be26 Add basic test coverage of new screen 2022-05-17 18:15:14 +09:00
Dean Herbert
30fdbc3de0 Add safety against calling GetStableImportPaths when path doesn't exist 2022-05-17 17:33:54 +09:00
Dean Herbert
4af1a788d1 Add locate stable button / screen 2022-05-17 15:55:53 +09:00
Dean Herbert
13e70eab51 Allow cancellation of count operations and bypassing interactive location logic 2022-05-17 15:55:53 +09:00
Dean Herbert
6448c97929 Allow retrieving count of available stable imports 2022-05-17 15:55:53 +09:00
Dean Herbert
218642c300 Update unsafe file write usages 2022-05-16 18:05:27 +09:00
Dean Herbert
33f024212f Fix realm refetch operations potentially being unsafe 
As seen in test failure https://github.com/ppy/osu/runs/6357384721?check_suite_focus=true.
 2022-05-10 21:04:30 +09:00
Dean Herbert
043599081b Split out INotificationOverlay to allow for easier testing 2022-04-18 20:14:01 +09:00
Dean Herbert
e315313266 Split out IDialogOverlay to allow for easier testing 2022-04-18 18:36:26 +09:00
Dean Herbert
975883da5c Move all usages of client.realm filename to a single const 2022-03-30 13:34:48 +09:00
Dean Herbert
0433d2fe6a Add safety to realm instance retrieval in RealmAccess 2022-03-29 11:40:58 +09:00
Dean Herbert
ea46e024c1 Avoid deadlock during migration failed exit process 2022-03-29 00:04:13 +09:00
Dean Herbert
b6ae0ebb6f Limit macOS specific log output to macOS platforms specifically 2022-03-28 23:46:07 +09:00
Dean Herbert
a98aac3bf2 Better inform users of migration failure reason when running Apply Silicon build 
As mentioned in https://github.com/ppy/osu/discussions/17409#discussioncomment-2445464?
 2022-03-28 17:34:36 +09:00
Dean Herbert
b04ca111c6 Allow realm subscriptions to be initiated from a non-update thread 2022-03-24 23:41:07 +09:00
Salman Ahmed
c2063f415d Remove unnecessary MaxCombo migration between unmapped database fields 2022-03-20 16:27:54 +03:00
Dean Herbert
6565c95b17 Remove unused variable 2022-03-08 18:19:54 +09:00
Dean Herbert
0718a55ad0 Add flow to allow recovery after running an older release (with a different realm database version) 
As brought up in https://github.com/ppy/osu/discussions/17148
 2022-03-08 18:15:28 +09:00
Dean Herbert
2a55c5e02e Add extension method to detect and isolate realm collection-level changes 2022-03-08 14:50:47 +09:00
Bartłomiej Dach
0fbc018a42
Remove redundant string interpolation prefixes 2022-03-03 20:21:09 +01:00
Dean Herbert
35f532fefa Add ability to watch properties via a RealmAccess helper method 2022-03-03 17:42:40 +09:00
Dean Herbert
7d11cfb301 Add detach mapping for BeatmapUserSettings 2022-03-01 20:12:59 +09:00
Dean Herbert
047e801da9 Store and retrieve offset from realm 2022-03-01 20:12:59 +09:00
Dean Herbert
acf8db13ac Store user settings to realm 2022-03-01 18:44:15 +09:00
Dean Herbert
9a117467b5 Add RealmAccess.WriteAsync method 2022-03-01 18:44:05 +09:00
Dean Herbert
7fa5842783 Add global statistics output for all realm reads/writes 2022-03-01 18:31:18 +09:00
Dean Herbert
334fe1f120 Add AsSplitQuery specification to avoid optimisation recommendation log messages 2022-02-15 15:23:17 +09:00
Dean Herbert
8ec28dc8bc Update OsuDbContext in line with EF changes 2022-02-15 15:19:50 +09:00
Dean Herbert
908c31c687 Update stream read operations to use new helper methods 2022-02-11 16:02:25 +09:00
Bartłomiej Dach
dc96c4888b
Add support for creating new blank difficulties 2022-02-02 21:55:33 +01:00
Bartłomiej Dach
a378e78ced
Fix RealmLive unnecessarily passing ID around 
Appears to have never been needed. When the `retrieveFromID` method was
created in 81b5717ae7, it didn't use the
`id` parameter for anything either.
 2022-02-01 21:35:12 +01:00
Dean Herbert
1a14ccc7ee Run EF migrations before migrating to realm 
Turns out that there are more than zero users that are upgrading from
old databases. I think we probably want to support this for now.

Tested against database in https://github.com/ppy/osu/discussions/16700
and one other I had locally, both work correctly.
 2022-01-29 23:13:23 +09:00
Dean Herbert
942ea896f1 Skip scores missing beatmaps during realm migration 2022-01-28 01:20:32 +09:00
Dean Herbert
81461be49f Skip beatmap imports where ruleset is not present in realm 
Closes #16651.

When a ruleset is not available, the `Find` call would return null. When
a null is passed to the constructor, `BeatmapInfo` would create an "osu"
ruleset, which tries to get stored to realm and fails on duplicate
primary key.

Probably need to add better safeties against this (or change that
constructor...) but this will fix the migration process.

Probably not serious enough to pull the build. This only affects
rulesets like karaoke which have custom beatmaps.
 2022-01-28 00:14:20 +09:00
Dan Balasescu
9ff2b9eb95
Merge pull request #16640 from peppy/realm-migration-always-delete 
Improve realm migration UX to give users a recovery path on error
 2022-01-27 15:43:14 +09:00
Dean Herbert
67ccb87992 Add exception message to discussion template url 2022-01-27 14:56:04 +09:00
Dean Herbert
fb081384e1 Add safety against zip creation potentially failing (probably can't but still) 2022-01-27 14:55:52 +09:00
Dean Herbert
465e7d29fe Avoid showing the external link warning 2022-01-27 14:53:11 +09:00
Dean Herbert
31abb372e5 Automatically zip and show the backup archive to the user 2022-01-27 14:52:58 +09:00
Dean Herbert
b745252962 Show notification when migration fails to give users a recovery path 2022-01-27 14:39:11 +09:00
Dean Herbert
deb5d75b5f Change migration process to always delete old EF database 
It is already backed up, so this is probably fine.
 2022-01-27 14:33:46 +09:00
Dean Herbert
587c0f965c Add more attempts to delete EF database 
Just noticed in passing. Probably best we do this since it was known to
fail on windows in some rare cases.
 2022-01-27 14:32:21 +09:00
Dean Herbert
f21e3d0d86 Block collection loading until realm migration has completed 2022-01-27 00:34:51 +09:00
Dan Balasescu
94d3e2c90c
Merge branch 'master' into fix-out-of-order-events-on-block-fail 2022-01-26 19:08:55 +09:00
Dan Balasescu
b1a23486c5
Merge pull request #16626 from peppy/fix-skin-section-realm-usage 
Refactor `SkinSection` to avoid unnecessary realm queries
 2022-01-26 19:01:15 +09:00
Dean Herbert
45636ce04b Remove collection ToLive helper method to avoid confusion 2022-01-26 18:26:37 +09:00
Dean Herbert
3aa681005b Skip importing scores which have no matching realm ruleset 
There's no real way to recover these unless we want to start importing
rulesets into realm. And that seems counter productive. This can only
happen if users don't have the dll present any more, and it was removed
far before realm was tracking rulesets (else it would have an
`Available=0` entry in realm to match).
 2022-01-26 18:04:54 +09:00
Dean Herbert
885fb92aad Move final empty result set sending to post-compact 2022-01-26 17:21:57 +09:00
Dean Herbert
791ea0308f Add flag to guard against deadlocks during blocking operations 2022-01-26 17:09:28 +09:00
Dean Herbert
11f0f3c17d Revert "Move final result set firing to before the update realm is disposed" 
This reverts commit 24bcba6418.
 2022-01-26 16:21:24 +09:00
Dean Herbert
24bcba6418 Move final result set firing to before the update realm is disposed 
Without this, if any registered callback attempts to access
`RealmAccess.Realm` when handling the empty set callback, it will
deadlock the game.
 2022-01-26 15:57:06 +09:00
Dean Herbert
5ea781faef Send unsubscribe actions to synchronization context for consistency and safety 2022-01-26 15:24:53 +09:00
Dean Herbert
91e0d1021f Merge branch 'master' into fix-out-of-order-events-on-block-fail 2022-01-26 15:21:10 +09:00
Dean Herbert
cd71ec0edd Remove ILive<> interface (and use abstract Live<> instead) 2022-01-26 13:38:56 +09:00
Dean Herbert
d37c3c463e Move statistics to static class 2022-01-26 13:29:12 +09:00
Dean Herbert
c7947b3489 Add statistics for Live usage 2022-01-26 12:42:24 +09:00
Dean Herbert
56b06f34f0 Fix RealmLive not refetching if update thread context was closed at some point 2022-01-26 12:32:34 +09:00
Dean Herbert
d76822b685 Avoid creating realm contexts or refetching when accessing RealmLive from the update thread 2022-01-26 12:32:34 +09:00
Dean Herbert
4a9f4eecba Use blocking calls to SynchronizationContext to guarantee order of execution 2022-01-25 20:49:52 +09:00
Dan Balasescu
d8270fe14f
Merge pull request #16604 from peppy/less-async-import 
Remove `Task` from the inner-most `Import` method in `RealmArchiveModelImporter`
 2022-01-25 16:47:07 +09:00
Dean Herbert
1bb1366c9f Fix notification reset events potentially arriving out of order if a block operation times out 2022-01-25 16:26:06 +09:00
Dan Balasescu
7f2e66298b
Merge branch 'master' into realm-subscription-tracking 2022-01-25 16:03:21 +09:00
Dean Herbert
778d2a71b4 Remove Task from the inner-most Import method in RealmArchiveModelImporter 
One of my pending work items for post-realm merge.

The lowest-level import task is no longer asynchronous, as we don't want
it to span multiple threads to allow easier interaction with realm.
Removing the `Task` spec simplifies a heap of usages.

Individual usages should decide whether they want to run the import
asynchronously, by either using an alternative override or spooling up a
thread themselves.
 2022-01-25 15:30:29 +09:00
Dean Herbert
5fb9b58c9b Add tracking of total subscriptions 2022-01-25 14:51:41 +09:00
Dean Herbert
9ad4d66e4d Merge branch 'realm-clean-up' into realm-block-timeout-assert-failure 2022-01-25 13:58:36 +09:00
Dean Herbert
a5c76a9647 Fix a few more cases of "context" terminology usage 2022-01-25 13:56:47 +09:00
Dean Herbert
d7342880f5 Update remaining cases of clashes with realm.Write and realm.RegisterForNotifications 2022-01-25 13:09:48 +09:00
Dean Herbert
e23b10e6a5 Update remaining cases of clashing variable name in realm.Run(realm.. 2022-01-25 13:04:05 +09:00
Dean Herbert
6f4c337a56 Fix a failed BlockAllOperations leaving update realm in unretrieved state 
If the operation timed out on..

```csharp
   throw new TimeoutException(@"Took too long to acquire lock");
```

..from an update thread, it would not restore the update context.

The next call would then fail on the assert that ensures a non-null
context in such cases.

Can add test coverage if required.
 2022-01-24 20:53:29 +09:00
Dean Herbert
f30894840c Update terminology to realm "instance" rather than "context" 
This matches the terminology used by realm themselves, which feels
better.
 2022-01-24 20:38:38 +09:00
Dean Herbert
6eb2c28e41 Rename RealmContextFactory to RealmAccess 2022-01-24 20:38:07 +09:00
Dan Balasescu
bb54ad9ad8
Merge pull request #16547 from peppy/realm-stable-subscriptions 
Fix realm subscriptions getting lost after a context recycle
 2022-01-24 20:20:04 +09:00
Dean Herbert
66c5d77d63 Allow realm migration to run again if interrupted halfway 2022-01-24 18:55:15 +09:00
Dean Herbert
9afa034296 Fix attempt to revive update thread realm context from non-update thread 2022-01-24 18:36:16 +09:00
Dean Herbert
9ff9611296 Merge branch 'master' into realm-stable-subscriptions 2022-01-24 18:25:47 +09:00
Dean Herbert
e3083c2477 Fix copy pasted xmldoc 2022-01-24 18:05:30 +09:00
Dean Herbert
bf5bf8d1fd Rename dictionaries to match methods 2022-01-24 17:58:53 +09:00
Dean Herbert
f4e7211ef1 Add xmldoc for RegisterForNotifications 2022-01-24 17:52:36 +09:00
Dean Herbert
abf14f0982 Lock unregistration for sanity 2022-01-24 17:46:53 +09:00
Dean Herbert
52cd906af6 Move context retrieval inside lock 2022-01-24 17:45:31 +09:00
Dean Herbert
cb319cebdb Refactor naming and add more comments to help understanding in RealmContextFactory subscription logic 2022-01-24 14:48:55 +09:00
Dean Herbert
40aa873190 Rename register methods to better explain their purpose 2022-01-24 14:37:36 +09:00
Dean Herbert
997c13f643 Add locking over realmSubscriptionsResetMap for sanity 2022-01-24 13:36:51 +09:00
Dean Herbert
deb1670862 Use Array.Empty instead of constructed list 2022-01-23 23:18:50 +09:00
Dean Herbert
249f0f9697
Add more lengthy comment explaining cyclic avoidance 
Co-authored-by: Bartłomiej Dach <dach.bartlomiej@gmail.com>
 2022-01-23 23:15:39 +09:00
Dean Herbert
5e7993c35a Post disposal to synchronisation context 2022-01-23 20:43:25 +09:00
Dean Herbert
61cef42be9 Proof of concept realm subscriptions via Register 2022-01-23 20:28:04 +09:00
Dean Herbert
f39ff1eacb Add unregistration on blocking 
This is the first part of the requirement of sending a `ChangeSet` event
to ensure correct state during blocking time
 2022-01-23 20:27:57 +09:00
Dean Herbert
70a120ea8a Add missing lock coverage when using subscriptionActions dictionary 2022-01-23 18:00:24 +09:00
Bartłomiej Dach
7e68371d28
Move log statement about migration completed closer to rest of migration code 2022-01-22 14:20:28 +01:00
Bartłomiej Dach
1b2cca4a0d
Merge branch 'master' into realm-migration-ui 2022-01-22 13:52:19 +01:00
Bartłomiej Dach
c99f227879
Remove no longer used resolved storage 2022-01-22 13:48:49 +01:00
Dean Herbert
6a076a684e Merge branch 'realm-context-use-update-when-feasible' into realm-stable-subscriptions 2022-01-22 13:14:25 +09:00
Dean Herbert
cdcf0947e6 Merge branch 'master' into realm-context-use-update-when-feasible 2022-01-22 12:09:18 +09:00
Dean Herbert
b23f4674b1
Update outdated exception message 
Co-authored-by: Salman Ahmed <frenzibyte@gmail.com>
 2022-01-22 12:02:18 +09:00
Dean Herbert
daed0b04dc Remove using statements 2022-01-22 11:54:40 +09:00
Dean Herbert
855ef3fa92 Create backup before any realm contexts are used 2022-01-22 11:52:46 +09:00
Dean Herbert
8ef50ff42d Add safety to ensure RealmLiveUnmanaged is not used with managed instances 2022-01-22 02:13:08 +09:00
Dean Herbert
ad3a01dc06 Use a more reliable method of reviving the update thread realm after blocking 2022-01-21 22:44:56 +09:00
Dean Herbert
3b11235d3c Fix potentially cyclic subscription registration if a subscription's delegate accesses Context 2022-01-21 20:45:10 +09:00
Dean Herbert
1f157d729d Update existing subscriptions to new style 
Fix missing detach calls in `MusicController`
 2022-01-21 20:05:03 +09:00
Dean Herbert
45aea9add5 Implement full subscription flow 2022-01-21 19:06:23 +09:00
Dean Herbert
18bf690a30 Add Register method for subscription purposes and update safeties 2022-01-21 19:06:18 +09:00
Dean Herbert
8f1dfa33a2 Merge branch 'master' into realm-context-use-update-when-feasible 2022-01-21 18:03:09 +09:00
Dean Herbert
81b5717ae7 Fix RealmLive failing to retrieve due to lack of refresh 2022-01-21 17:33:03 +09:00
Dean Herbert
d2655c0825 Fix RealmLive not necessarily being in refreshed state due to potentially using update context 2022-01-21 17:27:30 +09:00
Dean Herbert
114c9e8c1f Update all usages of CreateContext to use either Run or Write 2022-01-21 17:27:08 +09:00
Dean Herbert
da0a803e6c Add RealmContextFactory.Write helper method 2022-01-21 17:08:05 +09:00
Dean Herbert
a59105635e Make CreateContext private 2022-01-21 16:40:20 +09:00
Dean Herbert
dde10d1ba2 Remove unused IRealmFactory interface 2022-01-21 16:38:07 +09:00
Dean Herbert
5622d2ba4f Show realm migration progress at Loader 2022-01-21 14:56:49 +09:00
Dean Herbert
0c9eb3ad61 Add realm factory helper methods to run work on the correct context 
Avoids constructing a new `Realm` instance when called from the update
thread without worrying about disposal.
 2022-01-21 01:33:47 +09:00
Dean Herbert
7aad2780b1 Add retry logic for realm backup creation 2022-01-21 00:46:53 +09:00
Dean Herbert
079b2dfc42 Create backup of databases before opening contexts 
Attempt to avoid file IO issues.

Closes #16531.
 2022-01-21 00:46:53 +09:00
Dean Herbert
5df46d0a54 Remove all calls to Realm.Refresh to fix blocking overhead from subscriptions 
Turns out this is not required if realm is aware of a
`SynchronizationContext`. See
https://github.com/realm/realm-dotnet/discussions/2775#discussioncomment-2005412
for further reading.
 2022-01-20 20:23:18 +09:00
Dean Herbert
ccddf9b47d Avoid constructor overhead for realm BeatmapSetInfo parameterless constructor 2022-01-20 17:03:44 +09:00
Dan Balasescu
64fe7d8dd3
Merge pull request #16520 from peppy/refresh-after-migrate 
Improve realm migration performance further
 2022-01-20 14:38:23 +09:00
Dean Herbert
a27a3b308c Remove duplicate setters 2022-01-20 04:34:00 +09:00
Dean Herbert
fad66d7682 Backup collections alongside main databases when migrating to realm 2022-01-19 16:31:36 +09:00
Dean Herbert
aa93042aa3 Centralise backup code and also run on skin/settings migrations 2022-01-19 16:31:36 +09:00
Dean Herbert
fd5198d667 Avoid using parameterless constructors in migration code 
Minor performance improvement (less garbage).
 2022-01-19 16:22:18 +09:00
Dean Herbert
42736c9995 Add transactional committing of scores/beatmaps 
This helps slightly with performance, allows better monitoring via realm
studio, but most importantly greatly reduces filesize.

fully compacted:        109M
transaction size 100:   115M
transaction size 1000:  123M
transaction size 10000: 164M
single transaction:     183M

With a transaction size of 100 there is a performance reduction, so 1000
seems to be the best middle-ground.
 2022-01-19 16:08:48 +09:00
Dean Herbert
973836484c Avoid using a write context for EF migration 
This reduces a stall delay as EF tries to process changes over tracked
objects during disposal of the context.
 2022-01-19 15:56:58 +09:00
Dean Herbert
faec62be51 Force a realm refresh after migration 
This really shouldn't have much effect as it will be run in the first
`Update` method and is probably a noop (we are already pointing to the
newest version due to just performing writes), but seems like a safe addition.

In general `Realm.Refresh()` only really does anything when there's multithreaded
usage and notifications to be sent.
 2022-01-19 15:56:58 +09:00
Dan Balasescu
fdf65aa2ff
Merge pull request #16502 from peppy/realm/remove-ef-migrations 
Remove all EF migrations and helper methods
 2022-01-19 14:43:45 +09:00
Dan Balasescu
3407b34b11
Merge branch 'master' into realm/remove-ef-migrations 2022-01-19 14:01:34 +09:00
Dan Balasescu
0b1736def6
Merge branch 'master' into realm-integration/faster-migration 2022-01-19 13:54:14 +09:00
Dan Balasescu
9c8f7795b7
Merge pull request #16510 from peppy/optimise-set-detach-part-2 
Avoid file retrieval overhead when detaching `BeatmapSetInfo`
 2022-01-19 13:53:34 +09:00
Dan Balasescu
a0336c6fc2
Merge pull request #16495 from peppy/realm-integration/reinit-on-corrupt 
Add support for starting with a fresh realm database if the existing one is not usable
 2022-01-19 13:30:14 +09:00
Dan Balasescu
2556f1cb87
Merge branch 'master' into optimise-set-detach-part-2 2022-01-19 13:20:38 +09:00
Dan Balasescu
d663fcd045
Merge pull request #16508 from peppy/optimise-set-detach 
Create separate automapper configurations to reduce cyclic detach overhead
 2022-01-19 13:20:25 +09:00
Dean Herbert
d440197fb3 Merge branch 'master' into realm-integration/faster-migration 2022-01-19 12:56:42 +09:00
Dean Herbert
ad4d9789ad Merge branch 'master' into realm-integration/reinit-on-corrupt 2022-01-19 12:31:56 +09:00
Dan Balasescu
f6f44029e8
Merge pull request #16507 from peppy/more-ingore-rules 
Add some missing `IgnoredAttributes` to reduce automapper overhead
 2022-01-19 12:26:37 +09:00
Dean Herbert
fb558a3189 Merge branch 'master' into realm-integration/reinit-on-corrupt 2022-01-19 12:26:24 +09:00
Dan Balasescu
c52899b1fb Rename property 2022-01-19 11:56:44 +09:00
Dean Herbert
261b4988f9 Only catch RealmExceptions to avoid blocking the nested TimeoutException 2022-01-19 10:58:59 +09:00
Dean Herbert
195534a1d2 Only output "successful" messages when copy actually occurred 2022-01-19 10:31:13 +09:00
Dean Herbert
e1a35714be Add notification for debug builds when database migration occurs 2022-01-19 10:31:10 +09:00
Dean Herbert
04e9ffa966 Freshen some comments 2022-01-19 10:20:56 +09:00
Dean Herbert
64a023665e Avoid taking more than one backup per migration run 2022-01-19 10:20:56 +09:00
Dean Herbert
761e161eec Add comment explaining ignore rule 2022-01-19 09:44:21 +09:00
Dean Herbert
60b80c88b6 Avoid file retrieval overhead when detaching BeatmapSetInfo 
It seems that no usages of `BeatmapSetInfo` detaches require files - a
`WorkingBeatmap` is always obtained before doing further lookups.

Therefore we can omit this include unless the detaching object is a
`BeatmapInfo`. A refetch is performed when retrieving a
`WorkingBeatmap` to complete the equation.
 2022-01-19 00:49:18 +09:00
Dean Herbert
f2b151023e Create separate automapper configurations to reduce cyclic detach overhead 
This optimises the `BeatmapSetInfo` detach operation by avoiding
detaching `BeatmapSetInfo.Beatmaps[].BeatmapSetInfo` a second time over.
 2022-01-19 00:03:24 +09:00
Dean Herbert
67bf95bc91 Remove all usage of AuthorString 2022-01-18 23:30:40 +09:00
Dean Herbert
6b0bf38c93 Use a single EF context to avoid scores getting cascade deleted along the way 2022-01-18 20:47:53 +09:00
Dean Herbert
6ddd2d59d3 Remove EF helper methods 2022-01-18 20:44:26 +09:00
Dean Herbert
519f7e6ad2 Don't bother with removing from EF as the file is going to be deleted anyway 2022-01-18 19:17:47 +09:00
Dean Herbert
3b0977903b Use IQueryable directly to avoid insane overheads 2022-01-18 19:17:47 +09:00
Dean Herbert
49fcff190a
Merge branch 'master' into realm-integration/reinit-on-corrupt 2022-01-18 18:02:23 +09:00
Dean Herbert
7baada2fa6
Merge branch 'master' into realm-integration/safer-migration 2022-01-18 18:01:46 +09:00
Dean Herbert
246a4a4bfe Add support for starting with a fresh realm database if the existing one is not usable 
The most common scenario is switching between schema versions when
testing. This should alleviate the manual overhead of that for the
majority of cases.

For users, this will show a notification on startup if their database
was purged, similar to what we had with EF.
 2022-01-18 16:05:13 +09:00
Dean Herbert
9a43ed742b Update automapper spec in line with v11 
See https://docs.automapper.org/en/latest/11.0-Upgrade-Guide.html for
 more details.
 2022-01-18 15:23:28 +09:00
Dean Herbert
1b62a685f3 Merge branch 'master' into realm-integration/score-and-beatmaps 2022-01-18 15:07:59 +09:00
Dean Herbert
3429fd8768 Fix transaction scope and add even more logging 2022-01-18 14:47:00 +09:00
Dean Herbert
bf50a9b8f8 Also backup the realm database before migration 2022-01-18 14:30:41 +09:00
Dean Herbert
2b1c15b6cc Allow BlockAllOperations to be called from a non-update thread if update has never run 2022-01-18 14:30:32 +09:00
Dean Herbert
cf30d48721 Add more logging during migration process 2022-01-18 14:21:33 +09:00
Dean Herbert
798482c941 Create backups before deleting scores and beatmaps from EF database 2022-01-18 14:21:33 +09:00
Dean Herbert
b1a75ce480 Permanently delete client.db after migration completes 2022-01-18 14:17:43 +09:00
Dean Herbert
a714941f93 Rename EF variable to make reading code easier 2022-01-18 14:06:22 +09:00
Dean Herbert
a7db793d8c Specify realm pipe path location 
I can't confirm this works yet, since I'm not sure what the
preconditions are for the `.note` file to be created. What I can confirm
is that a `.note` file hasn't appeared in my game data directory yet.
 2022-01-18 11:43:23 +09:00
Dean Herbert
2f2c498477 Fix importer not considering that some EF beatmaps have no local metadata 2022-01-14 23:31:42 +09:00
Dean Herbert
2840a71dda Uncomment EF deletion lines in migrations 2022-01-13 18:28:00 +09:00
Dean Herbert
b610d2db12 Add EF to realm beatmap migration 2022-01-13 18:23:18 +09:00
Dean Herbert
45a23e5a43 Add EF to realm score migration 2022-01-13 18:23:18 +09:00
Dean Herbert
c61419dfe5 Fix scores not using correct filename/display strings 
I've updated all cases where we should have been using
`GetDisplayString()` anyway, but left the `ToString()` implementations
in place for safety. They should probably be removed in the future.
 2022-01-13 16:56:11 +09:00
Dean Herbert
dcc354aa7c Fix deleted scores not being cleaned up on next startup 2022-01-13 13:40:09 +09:00
Dean Herbert
7a81fe19f6 Bump realm schema version to allow upgrades 2022-01-13 13:28:46 +09:00
Dean Herbert
7baff18764 Add back PerformRead return safety by checking IsManaged status of returned data 2022-01-13 13:14:44 +09:00
Dean Herbert
085893c9b4 Fix stray bracket 2022-01-13 13:03:57 +09:00
Dean Herbert
e12025dd48 Cascade delete metadata when beatmaps are deleted 2022-01-12 17:49:11 +09:00
Dean Herbert
5f7365e8f3 Ensure scores are cleaned up alongside beatmap so they don't have a null reference 2022-01-12 17:49:11 +09:00
Dean Herbert
f24b2b1be3 Make copying detached changes to realm only exposed for BeatmapSet 
Also fixes remaining issues with the copy process.
 2022-01-12 17:49:11 +09:00
Dean Herbert
a4de0f93fa Move manager Update methods to be explicit to where they are still used by legacy code 
Also fixes skin hash repopulation being completely broken.
 2022-01-12 17:49:11 +09:00
Dean Herbert
580ad03f8d Combine mapper configurations and add more explanation about special cases 2022-01-12 17:49:11 +09:00
Salman Ahmed
6db3c32dd1 Handle automapper realm cyclic references via AfterMaps 
This may not be the cleanest solution, but there don't seem to be any
way towards this either.

- `UseDestinationValue` has been inherited by default as noted in
https://docs.automapper.org/en/stable/10.0-Upgrade-Guide.html#usedestinationvalue-is-now-inherited-by-default, and its behaviour in this case would be using the nested **managed** realm object for the destination member rather than creating an unmanaged version.

- `MaxDepth` already sets `PreserveReferences` so there's no point of using it.

- `MaxDepth` should probably not be set for all maps, only for those with
cyclic references, to avoid the expensive overhead of `PreserveReferences`, as mentioned in https://docs.automapper.org/en/stable/5.0-Upgrade-Guide.html#circular-references.

That aside, `MaxDepth` should actually only be set to `1` for
`BeatmapSetInfo` mapping, because we don't want AutoMapper to create a
nested instance of `BeatmapSetInfo` in each mapped/detached beatmap, but
for some reason, doing that will cause automapper to not map any beatmap
inside the set and leave it with 0 beatmaps.

While on the other hand, using `MaxDepth(2)` for `BeatmapSetInfo` works,
but creates an unused instance of a `BeatmapSetInfo` inside each mapped
beatmap, which may not be ideal.

For `BeatmapInfo`, it has to be `MaxDepth(2)`, in which the first
`BeatmapInfo` depth would be itself (when detaching a beatmap), and the
second would be nested beatmaps inside the mapped/detached
`BeatmapSetInfo` within the beatmap. (note that when detaching a beatmap
set, the unused instance of `BeatmapSetInfo` within each beatmap of that
beatmap set doesn't also have a list of unused beatmaps as one might expect from the depth specification, it surprisingly has 0 beatmaps)

This causes it to create an unused instance of `BeatmapInfo` in the beatmap set resembling the root mapped/detached beatmap, but that one might be inevitable.
 2022-01-12 17:49:11 +09:00
Dean Herbert
e0c59f4b3c Localise EF context factory usage to migration only 2022-01-12 17:49:11 +09:00
Dean Herbert
f986c3ebd4 Add basic write support via automapper 2022-01-12 17:49:11 +09:00
Dean Herbert
aaefd72c69 Handle ignored mappings locally in Detach configuration 2022-01-12 17:00:17 +09:00
Dean Herbert
2b8706b6ce Detach and reattach scores to make work 2022-01-12 17:00:16 +09:00
Dean Herbert
8461eaab46 BeatmapSetInfo detach support 2022-01-12 17:00:16 +09:00
Dean Herbert
fe8a5e867d Remove updated/removed flow method mapping 2022-01-12 17:00:16 +09:00
Dean Herbert
8d943b5709 Fix many shortcomings and compatibility issues with EF classes post-rename 2022-01-12 17:00:16 +09:00
Dean Herbert
00e3af3366 Update model manager and many related classes to get things compiling again 2022-01-12 17:00:00 +09:00
Dean Herbert
d70e292828 Remove old EF classes 2022-01-12 16:57:27 +09:00
Dean Herbert
2a4bee61dd Update many score-related classes to move closer to being able to persist to realm 2022-01-12 16:57:27 +09:00
Dean Herbert
3da762e145 Replace EF ScoreInfo with realm version 
May contain errors.
 2022-01-12 16:57:27 +09:00
Dean Herbert
b8cd3cdbbc Various updates to ruleset and primary key usages to move closer to realm support 2022-01-12 16:57:13 +09:00
Dean Herbert
3ecd889fef Replace EF RulesetStore with realm version 
Pass full EF context factory to `RealmContextFactory` for migration purposes
 2022-01-12 16:39:36 +09:00
Dean Herbert
618903c217 Rename realm to become imposter classes 2022-01-12 16:39:36 +09:00
Dean Herbert
0b6c4497bd Rename EF classes to allow for shit to hit the fan 2022-01-12 16:39:36 +09:00
Dan Balasescu
af9fad00c3 Merge branch 'master' into async-deadlock-safety 2022-01-10 16:11:09 +09:00
Dean Herbert
d072f1d08d Add mention to detach methods of only running once 2022-01-10 13:09:26 +09:00
Dean Herbert
00177a3ae1 Update usages to new naming 2022-01-06 22:54:43 +09:00
Dean Herbert
3ea7588a91 Update continuation usages to use GetCompletedResult 2022-01-06 22:53:07 +09:00
Dean Herbert
73b40e6833 Replace usage of .Result with .WaitSafelyForResult 2022-01-04 11:51:41 +09:00
Bartłomiej Dach
ca7303a50a
Fix online lookup cache not clearing completed task on early return 
The task not being cleared in the early return path would cause
`pendingRequestTask` to become stuck as a completed task, and
`queryValue()` would not recreate it due to the null check there,
therefore stalling all lookups forevermore until a game restart.
 2021-12-21 11:13:41 +01:00
Bartłomiej Dach
ee89d8643e
Extract abstract implementation of online lookup cache 2021-12-21 11:12:51 +01:00
Dean Herbert
04cfae9bde Fix "Random Skin" text not showing up correctly 2021-12-18 16:26:15 +09:00
Dean Herbert
a9dbcd92a1 Split out unmanaged implementation of RealmLive into its own class 2021-12-16 15:23:55 +09:00
Dean Herbert
441b7baa93 Provide a realm factory to usages of ToLive/RealmLive 2021-12-14 14:26:34 +09:00
Dean Herbert
f7c5a3f506 Use similar method of consuming OnlineID as done in beatmap classes 2021-12-10 18:34:31 +09:00
Dean Herbert
8d9c37a825 Merge branch 'master' into primary-key-consistency 2021-12-08 21:34:38 +09:00
Bartłomiej Dach
4278a320e4
Fix skin setting resetting every launch 
The reason this was happening was an unfortunate oversight in the
migration logic. The code that was attempting to parse the skin settings
as `int` was firing regardless of whether a skin migration from EF to
realm had already occurred. If it had occurred, the skin setting would
contain a GUID rather than an integer, and therefore fail to parse, and
therefore implicitly fallback to a EF skin ID of 0 which would be the
default skin.

Fix by not running the setting migrating logic at all when there are no
EF skins to migrate.
 2021-12-06 20:12:02 +01:00
Dean Herbert
ae3038ead4 Overwrite existing files if AddFile is called with an existing filename 2021-12-02 17:18:06 +09:00
Dean Herbert
65f0a80c97 Merge branch 'master' into realm-integration/skins-rebase 2021-12-02 13:37:20 +09:00
Bartłomiej Dach
8bef50cbba
Revert "Refactor migration precondition to read better" 
Realm cannot translate `.All()` LINQ queries.

This reverts commit 0e0e8c25e8.
 2021-12-01 19:39:44 +01:00
Dan Balasescu
685bdd522e Replace 'skin' in comments with 'instance' 2021-12-01 20:17:38 +09:00
Dan Balasescu
c225f8f84c
Merge branch 'master' into realm-subscribe-helper-methods 2021-12-01 20:15:21 +09:00
Dan Balasescu
af704dfe5b
Merge pull request #15878 from peppy/beatmap-lookup-cache 
Cache beatmap metadata lookups used by multiplayer
 2021-12-01 20:13:21 +09:00
Dean Herbert
bf5a186a2b Add early abort to avoid sending empty lookup requests 2021-12-01 17:47:16 +09:00
Dean Herbert
81f82c24c3 Use new API endpoint to do batch lookups 2021-12-01 17:45:41 +09:00
Dean Herbert
09817ff161 Add missing returns additional documentation to main method call 2021-12-01 15:27:10 +09:00
Dean Herbert
4306420922 Add extension methods to add extra safety to realm subscriptions 
Also adjusts the naming and documentation to make it (hopefully) easier
to understand what this method/process implies.
 2021-12-01 15:09:53 +09:00
Dean Herbert
0e0e8c25e8 Refactor migration precondition to read better 2021-12-01 12:48:49 +09:00
Dean Herbert
ef3d0ee0db Merge branch 'master' into realm-integration/skins-rebase 2021-11-30 19:59:59 +09:00
Dean Herbert
87883f1fe4 Add BeatmapLookupCache 2021-11-30 19:28:45 +09:00
Dan Balasescu
8fdb9ab4e5
Merge branch 'master' into realm-integration/live-queryable-fix 2021-11-30 18:54:14 +09:00
Dan Balasescu
daa7135381
Merge branch 'master' into realm-nested-context-creation-deadlock-fix 2021-11-30 18:01:00 +09:00
Dean Herbert
e83ada40f9 Merge branch 'realm-nested-context-creation-deadlock-fix' into realm-integration/skins-rebase 2021-11-30 15:56:34 +09:00
Dean Herbert
6f66ecd77b Move migrations to own file and add user skin choice config migration 2021-11-30 15:41:18 +09:00
Dean Herbert
a943089be2 Add support for migration of skins from EF to Realm 2021-11-30 15:17:26 +09:00
Dean Herbert
6bf9327228 Add linking property on SkinFileInfo to allow EF to understand the link post-rename 2021-11-30 15:17:16 +09:00
Dean Herbert
6b0999052f Merge branch 'master' into realm-integration/live-queryable-fix 2021-11-30 12:02:37 +09:00
Dean Herbert
f3f77fa053 Update missed xmldoc/comments 2021-11-30 11:56:53 +09:00
Dean Herbert
35d68d6ab0 Remove all optimisations from RealmLive 2021-11-30 11:47:29 +09:00
Dan Balasescu
823552a5e8
Merge pull request #15856 from peppy/simplify-replace-file 
Remove the ability to specify a filename in `ReplaceFile`
 2021-11-29 20:49:38 +09:00
Dan Balasescu
8a188b74b1
Merge pull request #15850 from peppy/realm-integration/misc-qol 
Apply a couple of realm related documentation/interface changes
 2021-11-29 18:58:15 +09:00
Dean Herbert
0bcfb8e199 Fix regression in implementation of fix 2021-11-29 18:52:29 +09:00
Dean Herbert
cd0c811ab1 Add the ability to call ToString on a RealmLive to get the underlying object's implementation 2021-11-29 18:28:25 +09:00
Dean Herbert
5db7cf23d3 Add pending deletion skin cleanup 2021-11-29 18:28:25 +09:00
Dean Herbert
e283379f0e Replace EF SkinInfo with realm implementation 2021-11-29 18:28:25 +09:00
Dean Herbert
348d1d0be9 Don't dispose fetched realm instance when using RealmLive.Value 
See https://github.com/realm/realm-dotnet/discussions/2734#discussioncomment-1705038 for reasoning.
 2021-11-29 18:28:25 +09:00
Dean Herbert
1dd5e1ca89 Remove RealmLive context re-fetch optimisation for now 2021-11-29 18:28:25 +09:00
Dean Herbert
c411a755c7 Merge branch 'realm-nested-context-creation-deadlock-fix' into realm-integration/compiled 2021-11-29 18:28:07 +09:00
Dean Herbert
29f1ad10ba Merge branch 'realm-integration/misc-qol' into realm-integration/compiled 2021-11-29 18:28:03 +09:00
Dean Herbert
1cfcd33b07 Merge branch 'simplify-replace-file' into realm-integration/compiled 2021-11-29 18:28:02 +09:00
Dean Herbert
23fded4a3a Fix potential oversight in semaphore release logic 2021-11-29 18:26:37 +09:00
Dean Herbert
a7e4e7be3a Remove the ability to specify a filename in ReplaceFile 2021-11-29 18:11:51 +09:00
Dean Herbert
f19cfcc82e Make readonly 2021-11-29 17:38:54 +09:00
Dean Herbert
673481ebc5 Use ThreadLocal to avoid potential threading issues 2021-11-29 17:38:42 +09:00
Dean Herbert
6dcc244e21 Fix RealmLive not working is data is attached to realm post-ToLive call 2021-11-29 17:34:58 +09:00
Dean Herbert
c98451a7bf Fix potential deadlock on nested context creation requests 2021-11-29 16:29:53 +09:00
Dean Herbert
cb8fa80352 Don't dispose fetched realm instance when using RealmLive.Value 
See https://github.com/realm/realm-dotnet/discussions/2734#discussioncomment-1705038 for reasoning.
 2021-11-29 14:25:43 +09:00
Dean Herbert
e40e5096ea Remove RealmLive context re-fetch optimisation for now 2021-11-29 14:25:43 +09:00
Dean Herbert
21e8ab8357 Return ILive from ToLive calls instead of realm-typed instance 2021-11-29 14:12:22 +09:00
Bartłomiej Dach
93b5aec23e
Fix realm migration failures when upgrading from old versions 
* `RealmRulesetSetting` was added in 2021.916.0.
* `RealmKeyBinding` was added in 2021.703.0.

Attempting to upgrade from older releases than the above would cause
migrations up to schema versions 10 and 11 to fail.
 2021-11-28 15:00:40 +01:00
Dan Balasescu
5e53f51c99
Merge pull request #15812 from peppy/realm-live-unmanaged-fetch-fix 
Avoid attempting to fetch a non-managed `RealmLive` instance from the realm backing
 2021-11-26 15:37:58 +09:00
Dean Herbert
40d1b97af1 Avoid attempting to fetch a non-managed RealmLive instance from the realm backing 
For compatibility reasons, we quite often convert completely unmanaged
instances to `ILive`s so they fit the required parameters of a property
or method call. This ensures such cases will not cause any issues when
trying to interact with the underlying data.

Originally I had this allowing write operations, but that seems a bit
unsafe (when performing a write one would assume that the underlying
data is being persisted, whereas in this case it is not). We can change
this if the requirements change in the future, but I think throwing is
the safest bet for now.
 2021-11-26 15:06:14 +09:00
Dean Herbert
13612c0d02 Add equality support to ILive types 2021-11-26 14:56:16 +09:00
Dan Balasescu
f712aeee01
Merge pull request #15795 from peppy/realm-integration/separate-download-flow 
Split out download logic from main manager classes
 2021-11-25 20:59:23 +09:00
Dan Balasescu
d3a4890c31
Merge branch 'master' into realm-integration/stable-export-flow 2021-11-25 19:06:18 +09:00
Dean Herbert
e2ebcf7a26 Remove unnecessary manager parameter 
Confused why I added this in the first place..
 2021-11-25 18:36:03 +09:00
Dean Herbert
eeccf836ec Remove unnecessary GameHost parameter 2021-11-25 17:42:41 +09:00
Dean Herbert
79459c1aeb Fix typo in class and variable names 2021-11-25 17:12:15 +09:00
Dean Herbert
ec9a09d5a4 Add missing licence headers 2021-11-25 16:56:19 +09:00
Dean Herbert
cc1b91e4bd Split out legacy model export logic into LegacyModelExporter classes 2021-11-25 16:41:12 +09:00
Dean Herbert
7488ccd5fe Update all models to implement IHasNamedFiles 2021-11-25 16:41:12 +09:00
Dan Balasescu
6d60725b31
Merge branch 'master' into realm-integration/stable-import-flow 2021-11-25 16:36:02 +09:00
Dean Herbert
a0fa030f55 Rename base class to LegacyModelImporter 2021-11-25 16:33:04 +09:00
Dean Herbert
2df793ca22 Inline individual importers to avoid unnecessary construction for singular import types 2021-11-25 15:44:04 +09:00
Dean Herbert
9dcb20a821 Rename Stable to Legacy and add xmldoc 2021-11-25 15:39:05 +09:00
Dean Herbert
6cab7b877d Move stable import handling into its own class 2021-11-25 15:36:58 +09:00
Dean Herbert
bcdb73e1e8 Clean up files last 2021-11-25 14:28:27 +09:00
Dean Herbert
315e126497 Add automatic clean-up call for RealmFileStore 2021-11-25 14:17:44 +09:00
Dean Herbert
fd321109da Remove unnecessary virtual specification on Refresh 2021-11-25 14:06:12 +09:00
Dan Balasescu
1865227e08
Merge pull request #15773 from peppy/remove-model-file-list-inits 
Initialise `IHasFiles<TFile>.Files` lists at construction time
 2021-11-24 21:05:36 +09:00
Dean Herbert
052ed8f930 Add online status to RealmBeatmapSet 2021-11-24 18:48:02 +09:00
Dean Herbert
ddbd4f9473 Merge branch 'master' into remove-model-file-list-inits 2021-11-24 18:39:01 +09:00
Dan Balasescu
c4b5a07f4b
Merge branch 'master' into realm-move-data-migration-to-context-factory 2021-11-24 14:40:26 +09:00
Dean Herbert
99a139dc98 Initialise all file lists at construction time (and remove setter) 2021-11-24 13:56:21 +09:00
Dan Balasescu
5a0a95f9d4 Trim whitespace 2021-11-24 13:48:59 +09:00
Dean Herbert
6aed41eacc
Merge branch 'master' into realm-move-data-migration-to-context-factory 2021-11-24 12:38:44 +09:00
Dean Herbert
63b09b356f Rewrite commments to read better 2021-11-24 12:37:09 +09:00
Dean Herbert
6cd1bfd94f Remove outdated comment 2021-11-24 12:35:05 +09:00
Bartłomiej Dach
0d409fa33e
Merge branch 'master' into realm-ruleset-keybinding-short-name 2021-11-23 20:18:58 +01:00
Dean Herbert
6b134359c9 Handle the case where the ruleset isn't found in the EF database any more 2021-11-23 19:15:52 +09:00
Dean Herbert
4149d458f0 Move realm data migrations inside the RealmContextFactory 2021-11-23 18:14:15 +09:00
Dean Herbert
d9917ee6c7 Merge branch 'realm-ruleset-setting-short-name' into realm-ruleset-keybinding-short-name 2021-11-23 18:13:26 +09:00
Dean Herbert
40cd998f99 Fix incorrect disposal 2021-11-23 18:13:05 +09:00
Dean Herbert
782191c410 Merge branch 'realm-ruleset-setting-short-name' into realm-ruleset-keybinding-short-name 2021-11-23 17:48:48 +09:00
Dean Herbert
b1b6723826 Add xmldoc and verbatim string markers 2021-11-23 17:47:43 +09:00
Dean Herbert
505fede44d Pass the full EF context rather than a legacy RulesetStore 2021-11-23 17:40:20 +09:00
Dean Herbert
d94b27a8a2 Switch realm ruleset key bindings to use ruleset's ShortName as key 2021-11-22 18:52:30 +09:00
Dean Herbert
329bae50b0 Switch realm ruleset configuration to use ruleset's ShortName as key 2021-11-22 18:52:00 +09:00
Dean Herbert
ca26b6c540 Provide RealmContextFactory with the EF RulesetStore for migration purposes 2021-11-22 18:51:44 +09:00
Dean Herbert
d2062ff97f Reformat realm migrations list for legibility 2021-11-22 18:47:18 +09:00
Dean Herbert
361cb78880 Fix realm applying migrations from one version too early 2021-11-22 18:46:46 +09:00
Dean Herbert
83b4625bd5 Replace existing cases with new helper method 2021-11-19 22:15:41 +09:00
Dean Herbert
eecf6ad558 Add IsManaged helper method to EF classes to match realm implementation 2021-11-19 21:56:06 +09:00
Dean Herbert
59e763467f Move StoragePath implementation to an extension method 2021-11-19 16:08:38 +09:00
Dean Herbert
726a0cc091 Merge branch 'master' into beatmap-collection-inteface-types 2021-11-16 15:31:10 +09:00
Dean Herbert
a4c11e8813 Use extension method to compare online IDs 2021-11-15 14:34:50 +09:00
Bartłomiej Dach
e66b637587
Merge branch 'master' into fix-android-score-imports 2021-11-12 11:40:15 +01:00
Dean Herbert
692e846acd Rename BeatmapSetInfo.OnlineBeatmapSetID to OnlineID to match interface 2021-11-12 17:52:44 +09:00
Dean Herbert
6a098a8634 Rename BeatmapInfo.OnlineBeatmapID to OnlineID to match interface 2021-11-12 17:46:24 +09:00
Dean Herbert
adf81d7fcd Add pathway to correctly handle stream-based imports which are not zip archives 2021-11-12 16:42:07 +09:00
Dean Herbert
ad8a710a69 Fix failed imports being incorrectly considered as successfully importing for notification purposes 2021-11-12 16:10:46 +09:00
Bartłomiej Dach
5ec8288508
Add GetDisplayString() extension to handle all model interface types globally 2021-11-09 13:53:07 +01:00
Dean Herbert
dfbc1f3394 Fix "conflicting" variable name 2021-11-09 17:42:03 +09:00
Dean Herbert
999d625e76 Fix realm migration potentially failing for users that haven't run osu! in a long time 
As reported at https://github.com/ppy/osu/discussions/15530.
 2021-11-09 14:51:10 +09:00
Dean Herbert
78aef9ce86
Merge branch 'master' into remove-stupid-weak-reference-bindable-events 2021-11-08 13:35:49 +09:00
Dean Herbert
89cc2523ef Fix incorrectly specified events 2021-11-06 22:31:49 +09:00
Dean Herbert
2881ce0f5a Merge branch 'master' into remove-stupid-weak-reference-bindable-events 2021-11-06 14:08:39 +09:00
Dean Herbert
9a7c75508e Merge branch 'master' into user-class-cleanup 2021-11-05 19:56:18 +09:00
Dean Herbert
54f72d68ca Revert weird event flow in model manager/importers 2021-11-05 19:12:49 +09:00
Salman Ahmed
771bcbf868 Rename one more clashing variable 2021-11-05 12:24:07 +03:00
Dean Herbert
6c385ccd29 Move second generic to abstract model downloader rather than interface type 2021-11-05 17:37:05 +09:00
Dean Herbert
99df37f32d Add input generic type to IModelDownloader 2021-11-05 17:02:45 +09:00
Dean Herbert
846c80f12c Rename remaining clashing variables to appease CI somewhere 2021-11-05 17:02:24 +09:00
Dean Herbert
7772a97cf7 Add helper function to get realm class names rather than hard-coding each time 2021-11-05 17:01:00 +09:00
Dean Herbert
1fe9bca819 Change ModelDownloader's requirement to an IModelImporter rather than IModelManager 2021-11-05 16:47:18 +09:00
Dean Herbert
b90f44493c Remove importer inheritance from IModelManager 
Now only exists in legacy implementations, to reduce inheritance
complexity of interfaces which are going to be used going forwards.
 2021-11-05 16:45:38 +09:00
Dean Herbert
7acc4a4708 Refactor realm migrations to be a touch more legible 2021-11-04 18:57:54 +09:00
Dean Herbert
777b2cbcc4 Add realm migration for new author storage 2021-11-04 18:57:54 +09:00
Dean Herbert
b9983add15 Rename User to APIUser and move to correct namespace 2021-11-04 18:21:31 +09:00
Dean Herbert
ca438a102d Fix beatmap paths including invalid slash/backslash characters 2021-11-03 16:46:05 +09:00
Dan Balasescu
c51349de57
Merge pull request #15358 from peppy/online-id-potentially-zero 
Add a note about `OnlineID` potentially being zero in non-autoincrement cases
 2021-11-01 15:54:08 +09:00
Dean Herbert
19feae4a8e
Merge branch 'master' into beatmap-refactor/download-tracker 2021-11-01 13:15:36 +09:00
Bartłomiej Dach
b0c4fa662d
Merge branch 'master' into long-online-id-score-info 2021-10-30 14:35:30 +02:00
Dean Herbert
aff216840f Add a note about OnlineID potentially being zero in non-autoincrement cases 2021-10-29 16:54:03 +09:00
Dean Herbert
3b095e1626 Merge branch 'master' into beatmap-refactor/download-tracker 2021-10-29 11:50:54 +09:00
Dean Herbert
34d4715220 Allow for long online IDs and implement in ScoreInfo 2021-10-29 11:48:36 +09:00
Dean Herbert
3838fe5c6a Fix typo in variable name 2021-10-29 11:39:15 +09:00
Dean Herbert
8d21f0b04b Add migration to reset conflicting scroll speed key bindings 2021-10-29 11:14:22 +09:00
Dean Herbert
add39fbd4f Merge branch 'master' into fix-skin-layout-editor-crash 2021-10-28 06:24:24 +09:00
Dean Herbert
617e6febb6 Refactor ModelDownloader to allow for different OnlineID matching 2021-10-27 21:00:46 +09:00
Dean Herbert
9015ac6ba8 Implement new version of download tracker 2021-10-27 21:00:46 +09:00
Dean Herbert
95837990f3 Apply some second-pass inspections that appeared after previous changes 2021-10-27 13:09:30 +09:00
Dean Herbert
6944151486 Apply batch fixing of built-in types using var 2021-10-27 13:04:41 +09:00
Dean Herbert
370775af68 Fix some surrounding code formatting 2021-10-26 14:05:07 +09:00
Dean Herbert
26cf5370c3 Remove unused reader parameter 2021-10-24 23:48:46 +09:00
Susko3
51c5d0aec7 Update to be in line with framework changes 2021-10-24 13:21:41 +02:00
Susko3
4a425b5c3e Merge remote-tracking branch 'upstream/master' into use-ShowFileInNativeExplorer 2021-10-23 10:01:24 +02:00
Dean Herbert
eef9949a0a Remove unnecessary branching around EF logic 2021-10-22 14:48:20 +09:00
Dean Herbert
59b7210efa Revert disallowing imports with no files 
While it is logical that we want this, from a testing perspective this is a bit of a nightmare to fix. Let's revisit at a later point in time.
 2021-10-21 13:43:42 +09:00
Dean Herbert
9e6e41d7c0 Add migration to reset and repopulate existing skin hashes 2021-10-20 17:56:45 +09:00
Dean Herbert
439e90fce3 Disallow archive imports with no hashable files 2021-10-20 17:45:06 +09:00
Dean Herbert
789c715f13 Add skin.ini write support to allow for more correct hashing 2021-10-20 17:45:06 +09:00
Dan Balasescu
08f3bc3f44
Merge pull request #15120 from peppy/realm-importer 
Implement realm beatmap importer
 2021-10-20 16:44:16 +09:00
Dean Herbert
da750a74fc Add xmldoc mention of valid OnlineID values 2021-10-20 15:24:27 +09:00
Dean Herbert
d3ab45084d Fix realm migration potentially failing from older releases 2021-10-19 14:19:58 +09:00
Dean Herbert
6904938dc1 Merge branch 'realm-indexed-online-id' into realm-importer 2021-10-18 16:14:37 +09:00
Dean Herbert
88a575462c Work around weird null inspection 2021-10-18 16:11:55 +09:00
Dean Herbert
2c5ba1d8e2 Change OnlineID to non-nullable to allow for indexing in Realm 2021-10-18 15:53:40 +09:00
Dean Herbert
3c4c9ab7a7 Move ICanAcceptFiles specification to IModelImporter<TModel> 2021-10-18 14:25:51 +09:00
Susko3
25f172fae8 Update usages of OpenInNativeExplorer to use ShowFileInNativeExplorer 
Also updates WrappedStorage to override ShowFileInNativeExplorer.
 2021-10-17 12:56:29 +02:00
Dean Herbert
49969ac328 Add beatmap import and file store tests 2021-10-15 17:07:34 +09:00
Dean Herbert
d95a62fa56 Add models and stores for beatmap manager requirements 2021-10-15 16:58:41 +09:00
Dan Balasescu
05150b34e5
Merge pull request #15117 from peppy/update-present-import-post-import 
Update one missed instance of `PresentImport`
 2021-10-15 16:57:47 +09:00
Dean Herbert
e9c3d09e0e Remove redundant specs 2021-10-15 16:27:16 +09:00
Dean Herbert
fdb734ac91 Also add spec directly to IModelImporter 2021-10-15 16:24:40 +09:00