Dean Herbert
7cb4e32c17
Add one more lock to appease CI
2022-06-29 22:45:19 +09:00
Dean Herbert
a3b4a515fc
Merge branch 'master' into realm-fix-async-write-after-disposal
2022-06-29 22:39:05 +09:00
Dean Herbert
32af4e41ea
Add back thread safety and locking as required
2022-06-29 20:56:01 +09:00
Dean Herbert
ecdb30d215
Fix one more case of collection modification during enumeration
...
https://sentry.ppy.sh/share/issue/a61c27b2a63a4a6aa80e75873f9d87ca/
2022-06-29 02:28:23 +09:00
Dean Herbert
e10ac45fd7
Remove probably redundant realmLock
...
As far as I can tell all accesses are safe due to update thread
guarantees. The only weird one may be async writes during a
`BlockAllOperations`, but the `Compact` loop should handle this quite
amicably.
2022-06-28 16:55:54 +09:00
Dan Balasescu
396e7fc166
Merge pull request #18862 from peppy/empty-beatmap-file-fix-2
...
Fix second case of empty beatmaps being reported to sentry as errors
2022-06-27 20:37:57 +09:00
Dean Herbert
f355c3a3c9
Merge pull request #18864 from peppy/fix-realm-audio-track-test-fail
...
Centralise and harden editor-ready-for-use check
2022-06-27 19:42:49 +09:00
Dean Herbert
83982d258d
Throw immediately if attempting to WriteAsync after disposed
2022-06-27 19:34:28 +09:00
Dean Herbert
c39c99bd43
Ensure all async writes are completed before realm is disposed
2022-06-27 19:21:05 +09:00
Dean Herbert
f6a61472c4
Fix occasional failure in realm test proceedings due to incorrect Debug.Assert
...
After a `BlockAllOperations`, the restoration of the `updateRealm`
instance is not instance. It is posted to a `SynchronizationContext`.
The assertion which has been removed in this commit was assuming it
would always be an immediate operation.
To ensure this works as expected, I've tracked the initialised state via
a new `bool`.
```csharp
System.TimeoutException : Attempting to block for migration took too long.
1) Host threw exception System.AggregateException: One or more errors occurred. (: )
---> NUnit.Framework.AssertionException: :
at osu.Framework.Logging.ThrowingTraceListener.Fail(String message1, String message2)
at System.Diagnostics.TraceInternal.Fail(String message, String detailMessage)
at System.Diagnostics.TraceInternal.TraceProvider.Fail(String message, String detailMessage)
at System.Diagnostics.Debug.Fail(String message, String detailMessage)
at osu.Game.Database.RealmAccess.BlockAllOperations() in /opt/buildagent/work/ecd860037212ac52/osu.Game/Database/RealmAccess.cs:line 813
at osu.Game.OsuGameBase.<>c__DisplayClass108_1.<Migrate>b__0() in /opt/buildagent/work/ecd860037212ac52/osu.Game/OsuGameBase.cs:line 449
at osu.Framework.Threading.ScheduledDelegate.RunTaskInternal()
at osu.Framework.Threading.Scheduler.Update()
at osu.Framework.Graphics.Drawable.UpdateSubTree()
at osu.Framework.Graphics.Containers.CompositeDrawable.UpdateSubTree()
```
https://teamcity.ppy.sh/buildConfiguration/Osu_Build/322?hideProblemsFromDependencies=false&hideTestsFromDependencies=false&expandBuildTestsSection=true
2022-06-27 18:01:12 +09:00
Dean Herbert
01487a1185
Add assertion on realm re-fetch being not-null
2022-06-27 15:52:45 +09:00
Dean Herbert
da61d0547f
Include archive name in import log output
2022-06-27 15:24:25 +09:00
Dean Herbert
31a447fda0
Update parameter discards
2022-06-24 21:26:19 +09:00
Dean Herbert
30b3973c9f
Difficulty cache invalidation flow
2022-06-24 21:02:38 +09:00
Bartłomiej Dach
26c5b59f6d
Replace usages of string.To{Lower,Upper}()
2022-06-24 11:57:45 +02:00
Dean Herbert
51268d0cc8
Throw on Wait failure in a few remaining cases
2022-06-23 15:28:21 +09:00
Dean Herbert
7ef8b7df5f
Add timeout for all ManualResetEventSlim.Wait invocations
...
Timeout values were taken as best-guesses of upper values we'd expect
from sane execution.
2022-06-23 14:46:00 +09:00
Dean Herbert
948c28f415
Fix collection modified during BlockAllOperations if any subscriptions have been established
2022-06-22 20:34:05 +09:00
Dan Balasescu
816fd338cb
Fix typo
2022-06-21 19:57:27 +09:00
Dean Herbert
f74b4ac277
Fix blocking overhead when calling WriteAsync
2022-06-21 17:15:25 +09:00
Dean Herbert
ba394f2831
Remove IPostsImports interface (and move to IModelImporter)
2022-06-20 18:53:48 +09:00
Dean Herbert
e732c5a2d7
Add PostImport method matching PreImport
2022-06-20 18:40:23 +09:00
Dean Herbert
45c5013f09
Remove default value of RealmArchiveModelImporter.HandledExtensions
...
Not used anywhere and probably not wanted ever.
2022-06-20 18:40:23 +09:00
Dean Herbert
dbae4c6f5a
PostImport -> PresentImport
2022-06-20 18:36:29 +09:00
Dean Herbert
4aa32038c3
Add note about stream provided to ImportTask being disposed implicitly
2022-06-20 15:38:44 +09:00
Dean Herbert
882e1c69e5
Move nested task back outside try-catch to avoid double-error
2022-06-20 15:36:44 +09:00
Dean Herbert
678b18dde6
Ensure any non-MemoryStream streams are closed as part of ImportTask
2022-06-20 15:29:56 +09:00
Dean Herbert
dd93c7359e
Update xmldoc for importFromArchive method
2022-06-20 15:25:43 +09:00
Dean Herbert
92f1a2958c
Rename Import(TModel) to ImportModel to differentiate from other import methods
2022-06-20 15:18:07 +09:00
Dean Herbert
a6c8a832aa
Remove Import(ArchiveReader) and redirect existing usages to Import(ImportTask)
2022-06-20 15:14:57 +09:00
Dan Balasescu
f8830c6850
Automated #nullable processing
2022-06-17 16:37:17 +09:00
Bartłomiej Dach
651862fee0
Merge branch 'master' into database-migration-reliability
2022-06-16 17:19:58 +02:00
Dean Herbert
bf6c6682bc
Move null to finally
2022-06-16 23:37:24 +09:00
Dean Herbert
7809566f16
Add explanatory comments
2022-06-16 23:31:49 +09:00
Dean Herbert
7b0fad6461
Null disposal token after use
2022-06-16 23:29:33 +09:00
Dean Herbert
448eee051a
Update filename to match new class name
2022-06-16 19:41:14 +09:00
Dean Herbert
ce3d3a967c
Reimplement missing methods
2022-06-16 19:05:25 +09:00
Dean Herbert
72c5b9009d
Remove local realm fields in manager classes
2022-06-16 18:56:53 +09:00
Dean Herbert
f3984d98e6
Remove RealmArchiveModelManager from hierarchy
2022-06-16 18:53:13 +09:00
Dean Herbert
1f3e1b2d97
Combine BeatmapManager and BeatmapModelManager into one class
2022-06-16 18:07:04 +09:00
Dean Herbert
4526f8c07d
Move database backup creation to async thread where possible
2022-06-16 17:01:17 +09:00
Dean Herbert
739a696467
Ensure reading of existing databases when making backups doesn't create a file
2022-06-16 16:48:06 +09:00
Dean Herbert
f23ddfe6cc
Move remaining realm classes out of Stores namespace
2022-06-15 21:56:00 +09:00
Dean Herbert
cca7633e59
Rename variables and remove unnecessary nested call to GetFullPath
2022-06-15 15:05:51 +09:00
Salman Ahmed
22c09ec893
Handle subdirectories during beatmap stable import
2022-06-15 07:33:21 +03:00
Dean Herbert
e96c5ce703
Move and correct xmldoc to interfaces
2022-06-14 20:00:16 +09:00
Dean Herbert
d73c0ca659
Remove unused methods from IModelImporter
...
All the xmldoc is already present in `RealmArchiveModelImporter` so
nothing is lost here.
2022-06-14 19:42:11 +09:00
Dean Herbert
1b4c89c418
Update realm to latest version
...
Contains minor changes to async usage in line with upstream API changes.
I believe a feedback issue we were seeing with offset changes (the only
component using async write flow) may have been resolved by these
upstream changes (see [release
notes](https://github.com/realm/realm-dotnet/releases/tag/10.14.0 )) but
am not investigating further just yet.
2022-06-03 14:16:10 +09:00
Dean Herbert
45841673f6
Update OnlineLookupCache to use async version of Perform call
2022-05-30 16:13:13 +09:00
Dean Herbert
4633eed78c
Merge branch 'master' into localisable-filter-items
2022-05-20 23:55:16 +09:00
Dean Herbert
70bd40ce44
Fix incorrect count of beatmaps available to import
2022-05-19 14:01:24 +09:00
Dean Herbert
1b7ec1be26
Add basic test coverage of new screen
2022-05-17 18:15:14 +09:00
Dean Herbert
30fdbc3de0
Add safety against calling GetStableImportPaths when path doesn't exist
2022-05-17 17:33:54 +09:00
Dean Herbert
4af1a788d1
Add locate stable button / screen
2022-05-17 15:55:53 +09:00
Dean Herbert
13e70eab51
Allow cancellation of count operations and bypassing interactive location logic
2022-05-17 15:55:53 +09:00
Dean Herbert
6448c97929
Allow retrieving count of available stable imports
2022-05-17 15:55:53 +09:00
Dean Herbert
218642c300
Update unsafe file write usages
2022-05-16 18:05:27 +09:00
Dean Herbert
33f024212f
Fix realm refetch operations potentially being unsafe
...
As seen in test failure https://github.com/ppy/osu/runs/6357384721?check_suite_focus=true .
2022-05-10 21:04:30 +09:00
Dean Herbert
043599081b
Split out INotificationOverlay to allow for easier testing
2022-04-18 20:14:01 +09:00
Dean Herbert
e315313266
Split out IDialogOverlay to allow for easier testing
2022-04-18 18:36:26 +09:00
Dean Herbert
975883da5c
Move all usages of client.realm filename to a single const
2022-03-30 13:34:48 +09:00
Dean Herbert
0433d2fe6a
Add safety to realm instance retrieval in RealmAccess
2022-03-29 11:40:58 +09:00
Dean Herbert
ea46e024c1
Avoid deadlock during migration failed exit process
2022-03-29 00:04:13 +09:00
Dean Herbert
b6ae0ebb6f
Limit macOS specific log output to macOS platforms specifically
2022-03-28 23:46:07 +09:00
Dean Herbert
a98aac3bf2
Better inform users of migration failure reason when running Apply Silicon build
...
As mentioned in https://github.com/ppy/osu/discussions/17409#discussioncomment-2445464 ?
2022-03-28 17:34:36 +09:00
Dean Herbert
b04ca111c6
Allow realm subscriptions to be initiated from a non-update thread
2022-03-24 23:41:07 +09:00
Salman Ahmed
c2063f415d
Remove unnecessary MaxCombo migration between unmapped database fields
2022-03-20 16:27:54 +03:00
Dean Herbert
6565c95b17
Remove unused variable
2022-03-08 18:19:54 +09:00
Dean Herbert
0718a55ad0
Add flow to allow recovery after running an older release (with a different realm database version)
...
As brought up in https://github.com/ppy/osu/discussions/17148
2022-03-08 18:15:28 +09:00
Dean Herbert
2a55c5e02e
Add extension method to detect and isolate realm collection-level changes
2022-03-08 14:50:47 +09:00
Bartłomiej Dach
0fbc018a42
Remove redundant string interpolation prefixes
2022-03-03 20:21:09 +01:00
Dean Herbert
35f532fefa
Add ability to watch properties via a RealmAccess helper method
2022-03-03 17:42:40 +09:00
Dean Herbert
7d11cfb301
Add detach mapping for BeatmapUserSettings
2022-03-01 20:12:59 +09:00
Dean Herbert
047e801da9
Store and retrieve offset from realm
2022-03-01 20:12:59 +09:00
Dean Herbert
acf8db13ac
Store user settings to realm
2022-03-01 18:44:15 +09:00
Dean Herbert
9a117467b5
Add RealmAccess.WriteAsync method
2022-03-01 18:44:05 +09:00
Dean Herbert
7fa5842783
Add global statistics output for all realm reads/writes
2022-03-01 18:31:18 +09:00
Dean Herbert
334fe1f120
Add AsSplitQuery specification to avoid optimisation recommendation log messages
2022-02-15 15:23:17 +09:00
Dean Herbert
8ec28dc8bc
Update OsuDbContext in line with EF changes
2022-02-15 15:19:50 +09:00
Dean Herbert
908c31c687
Update stream read operations to use new helper methods
2022-02-11 16:02:25 +09:00
Bartłomiej Dach
dc96c4888b
Add support for creating new blank difficulties
2022-02-02 21:55:33 +01:00
Bartłomiej Dach
a378e78ced
Fix RealmLive unnecessarily passing ID around
...
Appears to have never been needed. When the `retrieveFromID` method was
created in 81b5717ae7
2022-02-01 21:35:12 +01:00
Dean Herbert
1a14ccc7ee
Run EF migrations before migrating to realm
...
Turns out that there are more than zero users that are upgrading from
old databases. I think we probably want to support this for now.
Tested against database in https://github.com/ppy/osu/discussions/16700
and one other I had locally, both work correctly.
2022-01-29 23:13:23 +09:00
Dean Herbert
942ea896f1
Skip scores missing beatmaps during realm migration
2022-01-28 01:20:32 +09:00
Dean Herbert
81461be49f
Skip beatmap imports where ruleset is not present in realm
...
Closes #16651 .
When a ruleset is not available, the `Find` call would return null. When
a null is passed to the constructor, `BeatmapInfo` would create an "osu"
ruleset, which tries to get stored to realm and fails on duplicate
primary key.
Probably need to add better safeties against this (or change that
constructor...) but this will fix the migration process.
Probably not serious enough to pull the build. This only affects
rulesets like karaoke which have custom beatmaps.
2022-01-28 00:14:20 +09:00
Dan Balasescu
9ff2b9eb95
Merge pull request #16640 from peppy/realm-migration-always-delete
...
Improve realm migration UX to give users a recovery path on error
2022-01-27 15:43:14 +09:00
Dean Herbert
67ccb87992
Add exception message to discussion template url
2022-01-27 14:56:04 +09:00
Dean Herbert
fb081384e1
Add safety against zip creation potentially failing (probably can't but still)
2022-01-27 14:55:52 +09:00
Dean Herbert
465e7d29fe
Avoid showing the external link warning
2022-01-27 14:53:11 +09:00
Dean Herbert
31abb372e5
Automatically zip and show the backup archive to the user
2022-01-27 14:52:58 +09:00
Dean Herbert
b745252962
Show notification when migration fails to give users a recovery path
2022-01-27 14:39:11 +09:00
Dean Herbert
deb5d75b5f
Change migration process to always delete old EF database
...
It is already backed up, so this is probably fine.
2022-01-27 14:33:46 +09:00
Dean Herbert
587c0f965c
Add more attempts to delete EF database
...
Just noticed in passing. Probably best we do this since it was known to
fail on windows in some rare cases.
2022-01-27 14:32:21 +09:00
Dean Herbert
f21e3d0d86
Block collection loading until realm migration has completed
2022-01-27 00:34:51 +09:00
Dan Balasescu
94d3e2c90c
Merge branch 'master' into fix-out-of-order-events-on-block-fail
2022-01-26 19:08:55 +09:00
Dan Balasescu
b1a23486c5
Merge pull request #16626 from peppy/fix-skin-section-realm-usage
...
Refactor `SkinSection` to avoid unnecessary realm queries
2022-01-26 19:01:15 +09:00
Dean Herbert
45636ce04b
Remove collection ToLive helper method to avoid confusion
2022-01-26 18:26:37 +09:00
Dean Herbert
3aa681005b
Skip importing scores which have no matching realm ruleset
...
There's no real way to recover these unless we want to start importing
rulesets into realm. And that seems counter productive. This can only
happen if users don't have the dll present any more, and it was removed
far before realm was tracking rulesets (else it would have an
`Available=0` entry in realm to match).
2022-01-26 18:04:54 +09:00
Dean Herbert
885fb92aad
Move final empty result set sending to post-compact
2022-01-26 17:21:57 +09:00
Dean Herbert
791ea0308f
Add flag to guard against deadlocks during blocking operations
2022-01-26 17:09:28 +09:00
