Dean Herbert
1f3e1b2d97
Combine BeatmapManager and BeatmapModelManager into one class
2022-06-16 18:07:04 +09:00
Dean Herbert
4526f8c07d
Move database backup creation to async thread where possible
2022-06-16 17:01:17 +09:00
Dean Herbert
739a696467
Ensure reading of existing databases when making backups doesn't create a file
2022-06-16 16:48:06 +09:00
Dean Herbert
f23ddfe6cc
Move remaining realm classes out of Stores namespace
2022-06-15 21:56:00 +09:00
Dean Herbert
cca7633e59
Rename variables and remove unnecessary nested call to GetFullPath
2022-06-15 15:05:51 +09:00
Salman Ahmed
22c09ec893
Handle subdirectories during beatmap stable import
2022-06-15 07:33:21 +03:00
Dean Herbert
e96c5ce703
Move and correct xmldoc to interfaces
2022-06-14 20:00:16 +09:00
Dean Herbert
d73c0ca659
Remove unused methods from IModelImporter
...
All the xmldoc is already present in `RealmArchiveModelImporter` so
nothing is lost here.
2022-06-14 19:42:11 +09:00
Dean Herbert
1b4c89c418
Update realm to latest version
...
Contains minor changes to async usage in line with upstream API changes.
I believe a feedback issue we were seeing with offset changes (the only
component using async write flow) may have been resolved by these
upstream changes (see [release
notes](https://github.com/realm/realm-dotnet/releases/tag/10.14.0 )) but
am not investigating further just yet.
2022-06-03 14:16:10 +09:00
Dean Herbert
45841673f6
Update OnlineLookupCache to use async version of Perform call
2022-05-30 16:13:13 +09:00
Dean Herbert
4633eed78c
Merge branch 'master' into localisable-filter-items
2022-05-20 23:55:16 +09:00
Dean Herbert
70bd40ce44
Fix incorrect count of beatmaps available to import
2022-05-19 14:01:24 +09:00
Dean Herbert
1b7ec1be26
Add basic test coverage of new screen
2022-05-17 18:15:14 +09:00
Dean Herbert
30fdbc3de0
Add safety against calling GetStableImportPaths when path doesn't exist
2022-05-17 17:33:54 +09:00
Dean Herbert
4af1a788d1
Add locate stable button / screen
2022-05-17 15:55:53 +09:00
Dean Herbert
13e70eab51
Allow cancellation of count operations and bypassing interactive location logic
2022-05-17 15:55:53 +09:00
Dean Herbert
6448c97929
Allow retrieving count of available stable imports
2022-05-17 15:55:53 +09:00
Dean Herbert
218642c300
Update unsafe file write usages
2022-05-16 18:05:27 +09:00
Dean Herbert
33f024212f
Fix realm refetch operations potentially being unsafe
...
As seen in test failure https://github.com/ppy/osu/runs/6357384721?check_suite_focus=true .
2022-05-10 21:04:30 +09:00
Dean Herbert
043599081b
Split out INotificationOverlay to allow for easier testing
2022-04-18 20:14:01 +09:00
Dean Herbert
e315313266
Split out IDialogOverlay to allow for easier testing
2022-04-18 18:36:26 +09:00
Dean Herbert
975883da5c
Move all usages of client.realm filename to a single const
2022-03-30 13:34:48 +09:00
Dean Herbert
0433d2fe6a
Add safety to realm instance retrieval in RealmAccess
2022-03-29 11:40:58 +09:00
Dean Herbert
ea46e024c1
Avoid deadlock during migration failed exit process
2022-03-29 00:04:13 +09:00
Dean Herbert
b6ae0ebb6f
Limit macOS specific log output to macOS platforms specifically
2022-03-28 23:46:07 +09:00
Dean Herbert
a98aac3bf2
Better inform users of migration failure reason when running Apply Silicon build
...
As mentioned in https://github.com/ppy/osu/discussions/17409#discussioncomment-2445464 ?
2022-03-28 17:34:36 +09:00
Dean Herbert
b04ca111c6
Allow realm subscriptions to be initiated from a non-update thread
2022-03-24 23:41:07 +09:00
Salman Ahmed
c2063f415d
Remove unnecessary MaxCombo migration between unmapped database fields
2022-03-20 16:27:54 +03:00
Dean Herbert
6565c95b17
Remove unused variable
2022-03-08 18:19:54 +09:00
Dean Herbert
0718a55ad0
Add flow to allow recovery after running an older release (with a different realm database version)
...
As brought up in https://github.com/ppy/osu/discussions/17148
2022-03-08 18:15:28 +09:00
Dean Herbert
2a55c5e02e
Add extension method to detect and isolate realm collection-level changes
2022-03-08 14:50:47 +09:00
Bartłomiej Dach
0fbc018a42
Remove redundant string interpolation prefixes
2022-03-03 20:21:09 +01:00
Dean Herbert
35f532fefa
Add ability to watch properties via a RealmAccess helper method
2022-03-03 17:42:40 +09:00
Dean Herbert
7d11cfb301
Add detach mapping for BeatmapUserSettings
2022-03-01 20:12:59 +09:00
Dean Herbert
047e801da9
Store and retrieve offset from realm
2022-03-01 20:12:59 +09:00
Dean Herbert
acf8db13ac
Store user settings to realm
2022-03-01 18:44:15 +09:00
Dean Herbert
9a117467b5
Add RealmAccess.WriteAsync method
2022-03-01 18:44:05 +09:00
Dean Herbert
7fa5842783
Add global statistics output for all realm reads/writes
2022-03-01 18:31:18 +09:00
Dean Herbert
334fe1f120
Add AsSplitQuery specification to avoid optimisation recommendation log messages
2022-02-15 15:23:17 +09:00
Dean Herbert
8ec28dc8bc
Update OsuDbContext in line with EF changes
2022-02-15 15:19:50 +09:00
Dean Herbert
908c31c687
Update stream read operations to use new helper methods
2022-02-11 16:02:25 +09:00
Bartłomiej Dach
dc96c4888b
Add support for creating new blank difficulties
2022-02-02 21:55:33 +01:00
Bartłomiej Dach
a378e78ced
Fix RealmLive unnecessarily passing ID around
...
Appears to have never been needed. When the `retrieveFromID` method was
created in 81b5717ae7
2022-02-01 21:35:12 +01:00
Dean Herbert
1a14ccc7ee
Run EF migrations before migrating to realm
...
Turns out that there are more than zero users that are upgrading from
old databases. I think we probably want to support this for now.
Tested against database in https://github.com/ppy/osu/discussions/16700
and one other I had locally, both work correctly.
2022-01-29 23:13:23 +09:00
Dean Herbert
942ea896f1
Skip scores missing beatmaps during realm migration
2022-01-28 01:20:32 +09:00
Dean Herbert
81461be49f
Skip beatmap imports where ruleset is not present in realm
...
Closes #16651 .
When a ruleset is not available, the `Find` call would return null. When
a null is passed to the constructor, `BeatmapInfo` would create an "osu"
ruleset, which tries to get stored to realm and fails on duplicate
primary key.
Probably need to add better safeties against this (or change that
constructor...) but this will fix the migration process.
Probably not serious enough to pull the build. This only affects
rulesets like karaoke which have custom beatmaps.
2022-01-28 00:14:20 +09:00
Dan Balasescu
9ff2b9eb95
Merge pull request #16640 from peppy/realm-migration-always-delete
...
Improve realm migration UX to give users a recovery path on error
2022-01-27 15:43:14 +09:00
Dean Herbert
67ccb87992
Add exception message to discussion template url
2022-01-27 14:56:04 +09:00
Dean Herbert
fb081384e1
Add safety against zip creation potentially failing (probably can't but still)
2022-01-27 14:55:52 +09:00
Dean Herbert
465e7d29fe
Avoid showing the external link warning
2022-01-27 14:53:11 +09:00
Dean Herbert
31abb372e5
Automatically zip and show the backup archive to the user
2022-01-27 14:52:58 +09:00
Dean Herbert
b745252962
Show notification when migration fails to give users a recovery path
2022-01-27 14:39:11 +09:00
Dean Herbert
deb5d75b5f
Change migration process to always delete old EF database
...
It is already backed up, so this is probably fine.
2022-01-27 14:33:46 +09:00
Dean Herbert
587c0f965c
Add more attempts to delete EF database
...
Just noticed in passing. Probably best we do this since it was known to
fail on windows in some rare cases.
2022-01-27 14:32:21 +09:00
Dean Herbert
f21e3d0d86
Block collection loading until realm migration has completed
2022-01-27 00:34:51 +09:00
Dan Balasescu
94d3e2c90c
Merge branch 'master' into fix-out-of-order-events-on-block-fail
2022-01-26 19:08:55 +09:00
Dan Balasescu
b1a23486c5
Merge pull request #16626 from peppy/fix-skin-section-realm-usage
...
Refactor `SkinSection` to avoid unnecessary realm queries
2022-01-26 19:01:15 +09:00
Dean Herbert
45636ce04b
Remove collection ToLive helper method to avoid confusion
2022-01-26 18:26:37 +09:00
Dean Herbert
3aa681005b
Skip importing scores which have no matching realm ruleset
...
There's no real way to recover these unless we want to start importing
rulesets into realm. And that seems counter productive. This can only
happen if users don't have the dll present any more, and it was removed
far before realm was tracking rulesets (else it would have an
`Available=0` entry in realm to match).
2022-01-26 18:04:54 +09:00
Dean Herbert
885fb92aad
Move final empty result set sending to post-compact
2022-01-26 17:21:57 +09:00
Dean Herbert
791ea0308f
Add flag to guard against deadlocks during blocking operations
2022-01-26 17:09:28 +09:00
Dean Herbert
11f0f3c17d
Revert "Move final result set firing to before the update realm is disposed"
...
This reverts commit 24bcba6418
2022-01-26 16:21:24 +09:00
Dean Herbert
24bcba6418
Move final result set firing to before the update realm is disposed
...
Without this, if any registered callback attempts to access
`RealmAccess.Realm` when handling the empty set callback, it will
deadlock the game.
2022-01-26 15:57:06 +09:00
Dean Herbert
5ea781faef
Send unsubscribe actions to synchronization context for consistency and safety
2022-01-26 15:24:53 +09:00
Dean Herbert
91e0d1021f
Merge branch 'master' into fix-out-of-order-events-on-block-fail
2022-01-26 15:21:10 +09:00
Dean Herbert
cd71ec0edd
Remove ILive<> interface (and use abstract Live<> instead)
2022-01-26 13:38:56 +09:00
Dean Herbert
d37c3c463e
Move statistics to static class
2022-01-26 13:29:12 +09:00
Dean Herbert
c7947b3489
Add statistics for Live usage
2022-01-26 12:42:24 +09:00
Dean Herbert
56b06f34f0
Fix RealmLive not refetching if update thread context was closed at some point
2022-01-26 12:32:34 +09:00
Dean Herbert
d76822b685
Avoid creating realm contexts or refetching when accessing RealmLive from the update thread
2022-01-26 12:32:34 +09:00
Dean Herbert
4a9f4eecba
Use blocking calls to SynchronizationContext to guarantee order of execution
2022-01-25 20:49:52 +09:00
Dan Balasescu
d8270fe14f
Merge pull request #16604 from peppy/less-async-import
...
Remove `Task` from the inner-most `Import` method in `RealmArchiveModelImporter`
2022-01-25 16:47:07 +09:00
Dean Herbert
1bb1366c9f
Fix notification reset events potentially arriving out of order if a block operation times out
2022-01-25 16:26:06 +09:00
Dan Balasescu
7f2e66298b
Merge branch 'master' into realm-subscription-tracking
2022-01-25 16:03:21 +09:00
Dean Herbert
778d2a71b4
Remove Task from the inner-most Import method in RealmArchiveModelImporter
...
One of my pending work items for post-realm merge.
The lowest-level import task is no longer asynchronous, as we don't want
it to span multiple threads to allow easier interaction with realm.
Removing the `Task` spec simplifies a heap of usages.
Individual usages should decide whether they want to run the import
asynchronously, by either using an alternative override or spooling up a
thread themselves.
2022-01-25 15:30:29 +09:00
Dean Herbert
5fb9b58c9b
Add tracking of total subscriptions
2022-01-25 14:51:41 +09:00
Dean Herbert
9ad4d66e4d
Merge branch 'realm-clean-up' into realm-block-timeout-assert-failure
2022-01-25 13:58:36 +09:00
Dean Herbert
a5c76a9647
Fix a few more cases of "context" terminology usage
2022-01-25 13:56:47 +09:00
Dean Herbert
d7342880f5
Update remaining cases of clashes with realm.Write and realm.RegisterForNotifications
2022-01-25 13:09:48 +09:00
Dean Herbert
e23b10e6a5
Update remaining cases of clashing variable name in realm.Run(realm..
2022-01-25 13:04:05 +09:00
Dean Herbert
6f4c337a56
Fix a failed BlockAllOperations leaving update realm in unretrieved state
...
If the operation timed out on..
```csharp
throw new TimeoutException(@"Took too long to acquire lock");
```
..from an update thread, it would not restore the update context.
The next call would then fail on the assert that ensures a non-null
context in such cases.
Can add test coverage if required.
2022-01-24 20:53:29 +09:00
Dean Herbert
f30894840c
Update terminology to realm "instance" rather than "context"
...
This matches the terminology used by realm themselves, which feels
better.
2022-01-24 20:38:38 +09:00
Dean Herbert
6eb2c28e41
Rename RealmContextFactory to RealmAccess
2022-01-24 20:38:07 +09:00
Dan Balasescu
bb54ad9ad8
Merge pull request #16547 from peppy/realm-stable-subscriptions
...
Fix realm subscriptions getting lost after a context recycle
2022-01-24 20:20:04 +09:00
Dean Herbert
66c5d77d63
Allow realm migration to run again if interrupted halfway
2022-01-24 18:55:15 +09:00
Dean Herbert
9afa034296
Fix attempt to revive update thread realm context from non-update thread
2022-01-24 18:36:16 +09:00
Dean Herbert
9ff9611296
Merge branch 'master' into realm-stable-subscriptions
2022-01-24 18:25:47 +09:00
Dean Herbert
e3083c2477
Fix copy pasted xmldoc
2022-01-24 18:05:30 +09:00
Dean Herbert
bf5bf8d1fd
Rename dictionaries to match methods
2022-01-24 17:58:53 +09:00
Dean Herbert
f4e7211ef1
Add xmldoc for RegisterForNotifications
2022-01-24 17:52:36 +09:00
Dean Herbert
abf14f0982
Lock unregistration for sanity
2022-01-24 17:46:53 +09:00
Dean Herbert
52cd906af6
Move context retrieval inside lock
2022-01-24 17:45:31 +09:00
Dean Herbert
cb319cebdb
Refactor naming and add more comments to help understanding in RealmContextFactory subscription logic
2022-01-24 14:48:55 +09:00
Dean Herbert
40aa873190
Rename register methods to better explain their purpose
2022-01-24 14:37:36 +09:00
Dean Herbert
997c13f643
Add locking over realmSubscriptionsResetMap for sanity
2022-01-24 13:36:51 +09:00
Dean Herbert
deb1670862
Use Array.Empty instead of constructed list
2022-01-23 23:18:50 +09:00
Dean Herbert
249f0f9697
Add more lengthy comment explaining cyclic avoidance
...
Co-authored-by: Bartłomiej Dach <dach.bartlomiej@gmail.com>
2022-01-23 23:15:39 +09:00
Dean Herbert
5e7993c35a
Post disposal to synchronisation context
2022-01-23 20:43:25 +09:00
Dean Herbert
61cef42be9
Proof of concept realm subscriptions via Register
2022-01-23 20:28:04 +09:00
Dean Herbert
f39ff1eacb
Add unregistration on blocking
...
This is the first part of the requirement of sending a `ChangeSet` event
to ensure correct state during blocking time
2022-01-23 20:27:57 +09:00
Dean Herbert
70a120ea8a
Add missing lock coverage when using subscriptionActions dictionary
2022-01-23 18:00:24 +09:00
Bartłomiej Dach
7e68371d28
Move log statement about migration completed closer to rest of migration code
2022-01-22 14:20:28 +01:00
Bartłomiej Dach
1b2cca4a0d
Merge branch 'master' into realm-migration-ui
2022-01-22 13:52:19 +01:00
Bartłomiej Dach
c99f227879
Remove no longer used resolved storage
2022-01-22 13:48:49 +01:00
Dean Herbert
6a076a684e
Merge branch 'realm-context-use-update-when-feasible' into realm-stable-subscriptions
2022-01-22 13:14:25 +09:00
Dean Herbert
cdcf0947e6
Merge branch 'master' into realm-context-use-update-when-feasible
2022-01-22 12:09:18 +09:00
Dean Herbert
b23f4674b1
Update outdated exception message
...
Co-authored-by: Salman Ahmed <frenzibyte@gmail.com>
2022-01-22 12:02:18 +09:00
Dean Herbert
daed0b04dc
Remove using statements
2022-01-22 11:54:40 +09:00
Dean Herbert
855ef3fa92
Create backup before any realm contexts are used
2022-01-22 11:52:46 +09:00
Dean Herbert
8ef50ff42d
Add safety to ensure RealmLiveUnmanaged is not used with managed instances
2022-01-22 02:13:08 +09:00
Dean Herbert
ad3a01dc06
Use a more reliable method of reviving the update thread realm after blocking
2022-01-21 22:44:56 +09:00
Dean Herbert
3b11235d3c
Fix potentially cyclic subscription registration if a subscription's delegate accesses Context
2022-01-21 20:45:10 +09:00
Dean Herbert
1f157d729d
Update existing subscriptions to new style
...
Fix missing detach calls in `MusicController`
2022-01-21 20:05:03 +09:00
Dean Herbert
45aea9add5
Implement full subscription flow
2022-01-21 19:06:23 +09:00
Dean Herbert
18bf690a30
Add Register method for subscription purposes and update safeties
2022-01-21 19:06:18 +09:00
Dean Herbert
8f1dfa33a2
Merge branch 'master' into realm-context-use-update-when-feasible
2022-01-21 18:03:09 +09:00
Dean Herbert
81b5717ae7
Fix RealmLive failing to retrieve due to lack of refresh
2022-01-21 17:33:03 +09:00
Dean Herbert
d2655c0825
Fix RealmLive not necessarily being in refreshed state due to potentially using update context
2022-01-21 17:27:30 +09:00
Dean Herbert
114c9e8c1f
Update all usages of CreateContext to use either Run or Write
2022-01-21 17:27:08 +09:00
Dean Herbert
da0a803e6c
Add RealmContextFactory.Write helper method
2022-01-21 17:08:05 +09:00
Dean Herbert
a59105635e
Make CreateContext private
2022-01-21 16:40:20 +09:00
Dean Herbert
dde10d1ba2
Remove unused IRealmFactory interface
2022-01-21 16:38:07 +09:00
Dean Herbert
5622d2ba4f
Show realm migration progress at Loader
2022-01-21 14:56:49 +09:00
Dean Herbert
0c9eb3ad61
Add realm factory helper methods to run work on the correct context
...
Avoids constructing a new `Realm` instance when called from the update
thread without worrying about disposal.
2022-01-21 01:33:47 +09:00
Dean Herbert
7aad2780b1
Add retry logic for realm backup creation
2022-01-21 00:46:53 +09:00
Dean Herbert
079b2dfc42
Create backup of databases before opening contexts
...
Attempt to avoid file IO issues.
Closes #16531 .
2022-01-21 00:46:53 +09:00
Dean Herbert
5df46d0a54
Remove all calls to Realm.Refresh to fix blocking overhead from subscriptions
...
Turns out this is not required if realm is aware of a
`SynchronizationContext`. See
https://github.com/realm/realm-dotnet/discussions/2775#discussioncomment-2005412
for further reading.
2022-01-20 20:23:18 +09:00
Dean Herbert
ccddf9b47d
Avoid constructor overhead for realm BeatmapSetInfo parameterless constructor
2022-01-20 17:03:44 +09:00
Dan Balasescu
64fe7d8dd3
Merge pull request #16520 from peppy/refresh-after-migrate
...
Improve realm migration performance further
2022-01-20 14:38:23 +09:00
Dean Herbert
a27a3b308c
Remove duplicate setters
2022-01-20 04:34:00 +09:00
Dean Herbert
fad66d7682
Backup collections alongside main databases when migrating to realm
2022-01-19 16:31:36 +09:00
Dean Herbert
aa93042aa3
Centralise backup code and also run on skin/settings migrations
2022-01-19 16:31:36 +09:00
Dean Herbert
fd5198d667
Avoid using parameterless constructors in migration code
...
Minor performance improvement (less garbage).
2022-01-19 16:22:18 +09:00
Dean Herbert
42736c9995
Add transactional committing of scores/beatmaps
...
This helps slightly with performance, allows better monitoring via realm
studio, but most importantly greatly reduces filesize.
fully compacted: 109M
transaction size 100: 115M
transaction size 1000: 123M
transaction size 10000: 164M
single transaction: 183M
With a transaction size of 100 there is a performance reduction, so 1000
seems to be the best middle-ground.
2022-01-19 16:08:48 +09:00
Dean Herbert
973836484c
Avoid using a write context for EF migration
...
This reduces a stall delay as EF tries to process changes over tracked
objects during disposal of the context.
2022-01-19 15:56:58 +09:00
Dean Herbert
faec62be51
Force a realm refresh after migration
...
This really shouldn't have much effect as it will be run in the first
`Update` method and is probably a noop (we are already pointing to the
newest version due to just performing writes), but seems like a safe addition.
In general `Realm.Refresh()` only really does anything when there's multithreaded
usage and notifications to be sent.
2022-01-19 15:56:58 +09:00
Dan Balasescu
fdf65aa2ff
Merge pull request #16502 from peppy/realm/remove-ef-migrations
...
Remove all EF migrations and helper methods
2022-01-19 14:43:45 +09:00
Dan Balasescu
3407b34b11
Merge branch 'master' into realm/remove-ef-migrations
2022-01-19 14:01:34 +09:00
Dan Balasescu
0b1736def6
Merge branch 'master' into realm-integration/faster-migration
2022-01-19 13:54:14 +09:00
Dan Balasescu
9c8f7795b7
Merge pull request #16510 from peppy/optimise-set-detach-part-2
...
Avoid file retrieval overhead when detaching `BeatmapSetInfo`
2022-01-19 13:53:34 +09:00
Dan Balasescu
a0336c6fc2
Merge pull request #16495 from peppy/realm-integration/reinit-on-corrupt
...
Add support for starting with a fresh realm database if the existing one is not usable
2022-01-19 13:30:14 +09:00
Dan Balasescu
2556f1cb87
Merge branch 'master' into optimise-set-detach-part-2
2022-01-19 13:20:38 +09:00
Dan Balasescu
d663fcd045
Merge pull request #16508 from peppy/optimise-set-detach
...
Create separate automapper configurations to reduce cyclic detach overhead
2022-01-19 13:20:25 +09:00
Dean Herbert
d440197fb3
Merge branch 'master' into realm-integration/faster-migration
2022-01-19 12:56:42 +09:00
Dean Herbert
ad4d9789ad
Merge branch 'master' into realm-integration/reinit-on-corrupt
2022-01-19 12:31:56 +09:00
Dan Balasescu
f6f44029e8
Merge pull request #16507 from peppy/more-ingore-rules
...
Add some missing `IgnoredAttributes` to reduce automapper overhead
2022-01-19 12:26:37 +09:00
Dean Herbert
fb558a3189
Merge branch 'master' into realm-integration/reinit-on-corrupt
2022-01-19 12:26:24 +09:00
Dan Balasescu
c52899b1fb
Rename property
2022-01-19 11:56:44 +09:00
Dean Herbert
261b4988f9
Only catch RealmExceptions to avoid blocking the nested TimeoutException
2022-01-19 10:58:59 +09:00
Dean Herbert
195534a1d2
Only output "successful" messages when copy actually occurred
2022-01-19 10:31:13 +09:00
