Dean Herbert
f30c1a564f
Add basic setup for score migration
2023-06-13 01:40:43 +09:00
Bartłomiej Dach
ca25ac446b
Be slightly more specific with error message
2023-06-09 11:20:12 +02:00
Dean Herbert
53f935714e
Inline binary reading to avoid polluting RealmAccess with nested class
2023-06-09 17:34:27 +09:00
Dean Herbert
a9071e7afd
try-catch more
2023-06-09 17:33:59 +09:00
Dan Balasescu
67562a3856
Catch errors during score parsing
2023-06-09 14:35:29 +09:00
Dan Balasescu
facf7de053
Parse ScoreInfo.IsLegacyScore from replays
2023-06-08 21:38:10 +09:00
Dean Herbert
e0e013cca1
Fix incorrect realm schema comment
...
Co-authored-by: timiimit <32331609+timiimit@users.noreply.github.com>
2023-06-06 22:17:05 +09:00
Dean Herbert
878cdb2ed3
Bump schema version
2023-06-06 15:11:37 +09:00
Dean Herbert
18eb15bfa5
Gracefully handle failures in cleaning up pending file deletions
2023-05-31 19:39:43 +09:00
Bartłomiej Dach
2e81cae201
Move comment to more correct place
2023-05-30 23:18:34 +02:00
Dean Herbert
a0be526266
Adjust realm backup procedure to hard fail if running out of attempts
...
Previously, if the backup procedure failed, startup would continue and
the user's realm database may be deleted. I think in such a fail case
I'd rather the game didn't startup so the user gets in touch (or reads
the log files themselves) rather than potentially losing data.
2023-05-30 13:04:34 +09:00
Dean Herbert
79694897be
Ensure a potential exception from cleanupPendingDeletions doesn't mark realm corrupt
...
The whole restructure here is to move the nested call out of the
`try-catch`. I noticed this while looking at a corrupt database issue a
user reported (https://github.com/ppy/osu/discussions/23694 ).
It's not the first time we've seen a corrupt database error where the
"corrupt" version works just fine on a second attempt.
Maybe this isn't the issue and it's just a transitive file access violation
but it definitely feels like this should be fixed regardless.
2023-05-30 13:00:17 +09:00
Dean Herbert
c50ea89bc9
Simplify migration to not rely on old/dynamic schema
2023-02-08 14:24:06 +09:00
Cootz
4ba915268c
Change a comment into RealmAccess
2023-02-08 05:46:47 +03:00
PC
7e127dafe2
Update reference
2023-02-07 11:52:47 +03:00
Cootz
a1ee3df453
Improve local variable naming
2023-02-07 03:16:25 +03:00
Cootz
d23e787bc1
Update schema_version
2023-02-05 21:55:50 +03:00
Cootz
4598112586
Add OriginalBeatmapHash to ScoreInfo. Update db schema_version, migration
2023-02-05 21:46:38 +03:00
Dean Herbert
27c497145f
Fix the MOTHERLOAD of undetected issues that are now visible thanks to net6.0
2022-12-16 18:16:26 +09:00
Dean Herbert
42eafe318c
Remove various simple obsoletions
2022-10-28 13:20:58 +09:00
Dean Herbert
64ee210825
Add realm migration to update skin names
2022-09-18 00:12:43 +09:00
Dean Herbert
581a4d2d6d
Use APIBeatmap for realm migration requiring ShortName mapping
2022-09-15 16:58:58 +09:00
Dan Balasescu
d947a6cb59
Add Realm migration
2022-08-24 17:28:41 +09:00
Dean Herbert
e1e6be039a
Don't create destination stream if backup source doesn't exist
2022-08-17 18:20:47 +09:00
Dean Herbert
fca076b988
Fix edge case of realm backup cascading failure
2022-08-17 17:17:22 +09:00
Dean Herbert
11f38e539f
Rename property to LastLocalUpdate
2022-08-16 16:01:19 +09:00
Dean Herbert
c7db4a532c
Merge branch 'master' into store-modified-time
2022-08-16 15:53:51 +09:00
Dan Balasescu
98c7138803
Merge pull request #19586 from peppy/realm-versioning-for-debug
...
Version realm files for debug executions
2022-08-10 21:17:51 +09:00
Dean Herbert
c31e257a1f
Clean up pending deletion presets on startup
2022-08-07 15:16:33 +09:00
Dean Herbert
4544df5978
Leave client.realm around to handle pull requests without this change merged
2022-08-05 18:27:29 +09:00
Dean Herbert
ad3d00b1dc
Don't add version suffixes when running unit tests
2022-08-05 18:23:41 +09:00
Dean Herbert
3c84b1a389
Change order of application to use original client.realm last
2022-08-05 17:48:51 +09:00
Dean Herbert
84a3fbd25c
Version realm files for debug executions
...
To make it easier for developers to test out pull requests which bump
the realm schema version, realm files are now stored with the schema
version in the filename.
Note that this means any changes made to a newer version will not be
applied to previous ones.
2022-08-05 17:36:49 +09:00
Dean Herbert
d7a06abcab
Add BeatmapInfo.LastUpdate to track the time of local changes
2022-08-02 00:53:07 +09:00
Bartłomiej Dach
345f103119
Migrate mod preset column to use realm
2022-07-31 23:53:45 +02:00
Dean Herbert
6ad86ce5b7
Run collection import process asynchronously
...
Actually required to avoid deadlocking..
2022-07-31 01:06:57 +09:00
Dean Herbert
80ffa2cf20
Move collection database rather than deleting post-migration for safety
2022-07-31 00:54:00 +09:00
Dean Herbert
8e06d55960
Fix collection migration incorrectly running asynchronously
2022-07-31 00:53:39 +09:00
Dean Herbert
1669208a54
Add migration of existing collections database
2022-07-28 13:57:21 +09:00
Dean Herbert
8a0c8f5fd8
Fix some realm pieces not being cleaned up
2022-07-25 19:51:19 +09:00
Dean Herbert
94cd641fb4
Change migration to trigger reprocessing on every local beatmap
...
Was originally relying on the fact that this would be triggered due to a
null `LastOnlineUpdate`, but wouldn't cover the case of beatmaps with no
`OnlineID`.
2022-07-21 18:31:12 +09:00
Dean Herbert
d5e0dba9da
Change default value of StarRating to -1
2022-07-21 18:20:46 +09:00
Dean Herbert
68f28ff660
Add last applied version to RulesetInfo
2022-07-21 18:15:25 +09:00
Dean Herbert
30daa0fd44
Add ranked and submitted date storage and filtering
2022-07-19 19:55:56 +09:00
Dean Herbert
6ea380d649
Add new properties to BeatmapInfo to track online hash and updates
2022-07-19 17:57:01 +09:00
Dean Herbert
51f91fe62e
Update naming
2022-07-18 16:17:20 +09:00
Salman Ahmed
4e7156cee8
Store user country on databased scores
2022-07-16 06:39:05 +03:00
Dean Herbert
c8c79d2185
Standardise HasReplay implementation (and remove from persisting to realm)
2022-07-15 16:14:21 +09:00
Dean Herbert
8820ea4006
Add last played date to BeatmapInfo
2022-07-13 16:36:43 +09:00
Dean Herbert
33db508301
Add note regarding why the realmRetrievalLock return is done on the async thread
2022-07-05 16:23:10 +09:00
Dean Herbert
5adec2c738
Ensure blocking restoration only completes after update callback work is completed
2022-07-05 14:39:14 +09:00
Dean Herbert
1b98936328
Rename realm ThreadLocal to better convey what it's doing
...
Every time I looked at this code I have to re-learn what it's doing.
Changing these variable names should help quite a bit.
2022-07-05 13:38:25 +09:00
Dean Herbert
506409a9c4
Fix realm backup creation failing when run from RealmAccess constructor
...
At the point of construction, we are not on the update thread, but it
doesn't really matter at this point because there's no other usages.
2022-07-04 16:45:23 +09:00
Salman Ahmed
e28ee8bc7a
Count time spent upwards to display attempts correctly
2022-07-02 10:46:52 +03:00
Dean Herbert
93809a92d4
Fix clashing error messaging during realm block operations
2022-07-02 16:30:04 +09:00
Dean Herbert
4fd47b5fa0
Add more verbose logging to realm blocking process
2022-07-02 12:36:56 +09:00
Dean Herbert
7cb4e32c17
Add one more lock to appease CI
2022-06-29 22:45:19 +09:00
Dean Herbert
a3b4a515fc
Merge branch 'master' into realm-fix-async-write-after-disposal
2022-06-29 22:39:05 +09:00
Dean Herbert
32af4e41ea
Add back thread safety and locking as required
2022-06-29 20:56:01 +09:00
Dean Herbert
ecdb30d215
Fix one more case of collection modification during enumeration
...
https://sentry.ppy.sh/share/issue/a61c27b2a63a4a6aa80e75873f9d87ca/
2022-06-29 02:28:23 +09:00
Dean Herbert
e10ac45fd7
Remove probably redundant realmLock
...
As far as I can tell all accesses are safe due to update thread
guarantees. The only weird one may be async writes during a
`BlockAllOperations`, but the `Compact` loop should handle this quite
amicably.
2022-06-28 16:55:54 +09:00
Dean Herbert
83982d258d
Throw immediately if attempting to WriteAsync after disposed
2022-06-27 19:34:28 +09:00
Dean Herbert
c39c99bd43
Ensure all async writes are completed before realm is disposed
2022-06-27 19:21:05 +09:00
Dean Herbert
f6a61472c4
Fix occasional failure in realm test proceedings due to incorrect Debug.Assert
...
After a `BlockAllOperations`, the restoration of the `updateRealm`
instance is not instance. It is posted to a `SynchronizationContext`.
The assertion which has been removed in this commit was assuming it
would always be an immediate operation.
To ensure this works as expected, I've tracked the initialised state via
a new `bool`.
```csharp
System.TimeoutException : Attempting to block for migration took too long.
1) Host threw exception System.AggregateException: One or more errors occurred. (: )
---> NUnit.Framework.AssertionException: :
at osu.Framework.Logging.ThrowingTraceListener.Fail(String message1, String message2)
at System.Diagnostics.TraceInternal.Fail(String message, String detailMessage)
at System.Diagnostics.TraceInternal.TraceProvider.Fail(String message, String detailMessage)
at System.Diagnostics.Debug.Fail(String message, String detailMessage)
at osu.Game.Database.RealmAccess.BlockAllOperations() in /opt/buildagent/work/ecd860037212ac52/osu.Game/Database/RealmAccess.cs:line 813
at osu.Game.OsuGameBase.<>c__DisplayClass108_1.<Migrate>b__0() in /opt/buildagent/work/ecd860037212ac52/osu.Game/OsuGameBase.cs:line 449
at osu.Framework.Threading.ScheduledDelegate.RunTaskInternal()
at osu.Framework.Threading.Scheduler.Update()
at osu.Framework.Graphics.Drawable.UpdateSubTree()
at osu.Framework.Graphics.Containers.CompositeDrawable.UpdateSubTree()
```
https://teamcity.ppy.sh/buildConfiguration/Osu_Build/322?hideProblemsFromDependencies=false&hideTestsFromDependencies=false&expandBuildTestsSection=true
2022-06-27 18:01:12 +09:00
Dean Herbert
31a447fda0
Update parameter discards
2022-06-24 21:26:19 +09:00
Dean Herbert
948c28f415
Fix collection modified during BlockAllOperations if any subscriptions have been established
2022-06-22 20:34:05 +09:00
Dan Balasescu
816fd338cb
Fix typo
2022-06-21 19:57:27 +09:00
Dean Herbert
f74b4ac277
Fix blocking overhead when calling WriteAsync
2022-06-21 17:15:25 +09:00
Dan Balasescu
f8830c6850
Automated #nullable processing
2022-06-17 16:37:17 +09:00
Dean Herbert
4526f8c07d
Move database backup creation to async thread where possible
2022-06-16 17:01:17 +09:00
Dean Herbert
739a696467
Ensure reading of existing databases when making backups doesn't create a file
2022-06-16 16:48:06 +09:00
Dean Herbert
f23ddfe6cc
Move remaining realm classes out of Stores namespace
2022-06-15 21:56:00 +09:00
Dean Herbert
1b4c89c418
Update realm to latest version
...
Contains minor changes to async usage in line with upstream API changes.
I believe a feedback issue we were seeing with offset changes (the only
component using async write flow) may have been resolved by these
upstream changes (see [release
notes](https://github.com/realm/realm-dotnet/releases/tag/10.14.0 )) but
am not investigating further just yet.
2022-06-03 14:16:10 +09:00
Dean Herbert
218642c300
Update unsafe file write usages
2022-05-16 18:05:27 +09:00
Dean Herbert
33f024212f
Fix realm refetch operations potentially being unsafe
...
As seen in test failure https://github.com/ppy/osu/runs/6357384721?check_suite_focus=true .
2022-05-10 21:04:30 +09:00
Dean Herbert
975883da5c
Move all usages of client.realm filename to a single const
2022-03-30 13:34:48 +09:00
Dean Herbert
0433d2fe6a
Add safety to realm instance retrieval in RealmAccess
2022-03-29 11:40:58 +09:00
Dean Herbert
b04ca111c6
Allow realm subscriptions to be initiated from a non-update thread
2022-03-24 23:41:07 +09:00
Dean Herbert
6565c95b17
Remove unused variable
2022-03-08 18:19:54 +09:00
Dean Herbert
0718a55ad0
Add flow to allow recovery after running an older release (with a different realm database version)
...
As brought up in https://github.com/ppy/osu/discussions/17148
2022-03-08 18:15:28 +09:00
Bartłomiej Dach
0fbc018a42
Remove redundant string interpolation prefixes
2022-03-03 20:21:09 +01:00
Dean Herbert
35f532fefa
Add ability to watch properties via a RealmAccess helper method
2022-03-03 17:42:40 +09:00
Dean Herbert
047e801da9
Store and retrieve offset from realm
2022-03-01 20:12:59 +09:00
Dean Herbert
acf8db13ac
Store user settings to realm
2022-03-01 18:44:15 +09:00
Dean Herbert
9a117467b5
Add RealmAccess.WriteAsync method
2022-03-01 18:44:05 +09:00
Dean Herbert
7fa5842783
Add global statistics output for all realm reads/writes
2022-03-01 18:31:18 +09:00
Dean Herbert
885fb92aad
Move final empty result set sending to post-compact
2022-01-26 17:21:57 +09:00
Dean Herbert
791ea0308f
Add flag to guard against deadlocks during blocking operations
2022-01-26 17:09:28 +09:00
Dean Herbert
11f0f3c17d
Revert "Move final result set firing to before the update realm is disposed"
...
This reverts commit 24bcba6418
2022-01-26 16:21:24 +09:00
Dean Herbert
24bcba6418
Move final result set firing to before the update realm is disposed
...
Without this, if any registered callback attempts to access
`RealmAccess.Realm` when handling the empty set callback, it will
deadlock the game.
2022-01-26 15:57:06 +09:00
Dean Herbert
5ea781faef
Send unsubscribe actions to synchronization context for consistency and safety
2022-01-26 15:24:53 +09:00
Dean Herbert
91e0d1021f
Merge branch 'master' into fix-out-of-order-events-on-block-fail
2022-01-26 15:21:10 +09:00
Dean Herbert
4a9f4eecba
Use blocking calls to SynchronizationContext to guarantee order of execution
2022-01-25 20:49:52 +09:00
Dean Herbert
1bb1366c9f
Fix notification reset events potentially arriving out of order if a block operation times out
2022-01-25 16:26:06 +09:00
Dan Balasescu
7f2e66298b
Merge branch 'master' into realm-subscription-tracking
2022-01-25 16:03:21 +09:00
Dean Herbert
5fb9b58c9b
Add tracking of total subscriptions
2022-01-25 14:51:41 +09:00
Dean Herbert
9ad4d66e4d
Merge branch 'realm-clean-up' into realm-block-timeout-assert-failure
2022-01-25 13:58:36 +09:00
Dean Herbert
a5c76a9647
Fix a few more cases of "context" terminology usage
2022-01-25 13:56:47 +09:00
Dean Herbert
6f4c337a56
Fix a failed BlockAllOperations leaving update realm in unretrieved state
...
If the operation timed out on..
```csharp
throw new TimeoutException(@"Took too long to acquire lock");
```
..from an update thread, it would not restore the update context.
The next call would then fail on the assert that ensures a non-null
context in such cases.
Can add test coverage if required.
2022-01-24 20:53:29 +09:00
Dean Herbert
f30894840c
Update terminology to realm "instance" rather than "context"
...
This matches the terminology used by realm themselves, which feels
better.
2022-01-24 20:38:38 +09:00
