Shivam
66e61aacff
Logger now shows the actual path of the destination
...
Forgot to change this while changing the param from string to Storage
2020-07-02 00:32:09 +02:00
Shivam
c3cd2a74f5
Move general purpose migration to MigratableStorage
2020-07-01 22:57:16 +02:00
smoogipoo
cdcad94e9f
Handle exception thrown due to custom stoage on startup
2020-07-01 17:47:29 +09:00
smoogipoo
ab15b6031d
Update with framework-side storage changes
2020-07-01 17:12:07 +09:00
Shivam
47a732ef60
Address review comments
...
Now asserting instead of an if-statement, change cast from OsuStorage to MigratableStorage and make internal virtual properties protected.
2020-06-24 23:01:56 +02:00
Shivam
af11340849
Fix nullref exceptions and redundant explicit type
2020-06-24 02:13:28 +02:00
Shivam
c32ef5e718
Address formatting issues
2020-06-24 00:37:29 +02:00
Shivam
8e8458ab8f
make migrate public abstract in base and override
2020-06-23 23:58:28 +02:00
Shivam
a47d34f1db
make ignore properties protected virtual get-only in base
2020-06-23 23:34:26 +02:00
Shivam
a899c754f1
Remove whitespace at the end of xmldoc line
2020-06-22 13:03:24 +02:00
Shivam
a94dcc4923
Add xmldoc to MigratableStorage
2020-06-22 12:59:38 +02:00
Shivam
6b14079c0a
InspectCode changes
2020-06-22 12:43:01 +02:00
Shivam
f878388d57
Fix TestMigrationToSeeminglyNestedTarget failing
2020-06-22 11:56:14 +02:00
Shivam
21774b8967
Move static properties to parent class and inherit OsuStorage from it
2020-06-22 11:38:50 +02:00
Shivam
dd9697032c
Introduce new class MigratableStorage
2020-06-16 17:39:20 +02:00
smoogipoo
092f5b6521
Fix incorrect reference + simplify
2020-06-03 17:41:05 +09:00
smoogipoo
8aa8d2c880
Resolve NREs
2020-06-03 16:59:37 +09:00
Dean Herbert
6904d5d247
Remove unnecessary override
2020-05-25 13:12:53 +09:00
Dean Herbert
1977affe7e
Fix OpenInNativeExplorer not working correctly for wrapped storages
2020-05-25 09:27:11 +09:00
Dean Herbert
6ca102bc3f
Attempt delete operations more than once
2020-05-15 13:19:03 +09:00
Dean Herbert
9ed0e8891b
Merge branch 'fix-nested-folder-migration-endless-copy' into migration-ui
2020-05-15 13:04:16 +09:00
Dean Herbert
0690d81bbb
Add protection against migrating to a nested folder
2020-05-15 10:40:26 +09:00
Dean Herbert
25bbb02999
Throw better exceptions from OsuStorage
2020-05-15 10:28:03 +09:00
Dean Herbert
364aa5aa12
Add protection against migrating to a nested folder
2020-05-14 22:45:03 +09:00
Dean Herbert
827d75b152
Revert "Add protection against migrating to a nested folder"
...
This reverts commit ef8375b442
2020-05-14 22:44:27 +09:00
Dean Herbert
ef8375b442
Add protection against migrating to a nested folder
2020-05-14 22:42:42 +09:00
Dean Herbert
06f507496a
Delete migration source if no files exist after completion
2020-05-14 17:41:55 +09:00
Dean Herbert
cb0b25ac55
Throw better exceptions from OsuStorage
2020-05-14 17:41:55 +09:00
Dean Herbert
ad1d050fb4
Throw exception on copy timeout
2020-05-13 20:29:15 +09:00
Dean Herbert
49e616b7e5
Also check for directory presence before migrating
2020-05-13 20:19:14 +09:00
Dean Herbert
9de216f1b4
Merge branch 'master' into migration-backend
2020-05-13 20:15:22 +09:00
Huo Yaoyuan
f07d95ac59
Use IReadOnlyList<T> for TypedListConverter.
2020-05-12 21:49:55 +08:00
Huo Yaoyuan
f7d26688f4
Merge branch 'master' into json-converter
2020-05-12 21:43:02 +08:00
Dean Herbert
e4a23b3e7d
Fix ignored excluding more than top level
2020-05-12 12:39:04 +09:00
Dean Herbert
984f27c107
Add simple retry logic on file copy failure (may be in use)
2020-05-11 21:39:44 +09:00
Dean Herbert
a11be07bb1
Move log storage location after migration complete
2020-05-11 21:39:44 +09:00
Dean Herbert
36640f5dda
Merge branch 'master' into migration-backend
2020-05-11 18:52:36 +09:00
Dean Herbert
384862d48b
Fix incorrect relative paths when using GetStorageForDirectory
2020-05-10 13:20:13 +09:00
Dean Herbert
bbebd26efb
Use DirectoryInfo in more places
2020-05-09 20:13:31 +09:00
Huo Yaoyuan
80a193a616
Use IEnumerable<T> for TypedListConverter.
2020-05-09 17:41:59 +08:00
Huo Yaoyuan
74cbe9306c
Use strongly-typed JsonConerter.
2020-05-09 16:39:46 +08:00
Dean Herbert
7a2020fd45
User copy operation instead of move
2020-05-07 22:12:24 +09:00
Dean Herbert
49a03f1c06
Add basic blocking migration, move not copy
2020-05-07 22:11:57 +09:00
Dean Herbert
7781408643
Update in line with framework storage changes
2020-05-06 18:28:24 +09:00
smoogipoo
d27ca725f9
Use IEnumerable<T> instead
2020-04-22 09:04:44 +09:00
smoogipoo
a702a521f8
Fix not being able to serialise converted beatmaps
2020-04-22 09:04:44 +09:00
smoogipoo
f6f5de7ad1
Allow LineBufferedReader to keep underlying stream open
2020-03-31 10:13:50 +09:00
Dean Herbert
022465f546
Add encoding and import support
2020-03-24 14:51:52 +09:00
Dean Herbert
68ebe98fde
Remove unused GetUnderlyingStream method
2020-03-24 14:08:25 +09:00
Huo Yaoyuan
b14c7bd171
Use type switch in SearializationWriter.
2020-01-27 21:49:34 +08:00
Huo Yaoyuan
40b43b85f1
CA1715: use prefix for generic parameters.
2019-12-17 13:00:25 +08:00
Huo Yaoyuan
e46f6627e4
CA1052: make type static.
2019-12-17 13:00:13 +08:00
Huo Yaoyuan
31cc0d13da
Use 'out var'.
2019-11-12 19:55:26 +08:00
Dean Herbert
35be8f9dfb
Share framework file-exclusion function
2019-10-30 19:34:17 +09:00
Bartłomiej Dach
4b84564f47
Switch casing comparison mode to ordinal
...
Switch from InvariantCultureIgnoreCase to OrdinalIgnoreCase when
checking file paths in archives for substrings indicating the file can
be ignored for performance gains.
Co-Authored-By: Dan Balasescu <smoogipoo@smgi.me>
2019-10-11 09:24:41 +02:00
Bartłomiej Dach
57bfa18359
Filter out OS-generated files from archives
...
Add a filename ignore list to ZipArchiveReader to filter out superfluous
OS-generated files from archives during the import process. In addition
to decreasing the size of files imported this allows imports of some
incorrectly-constructed archives. An example is the case of having
a __MACOSX directory next to a single directory with the actual files -
filtering out the former at ZipArchiveReader allows the fallback added
in #6170 to work.
2019-10-10 23:54:23 +02:00
Dean Herbert
6268bbcfc8
Merge branch 'master' into beatmap-parsing-fallback-v2
2019-10-03 15:12:21 +08:00
Dean Herbert
d0a4e1e3c2
Force a checksum check before skipping FileStore copy op
2019-09-20 15:00:27 +09:00
Bartłomiej Dach
86588778b1
Implement fallback decoder registration
...
After the preparatory introduction of LineBufferedReader, it is now
possible to introduce registration of fallback decoders that won't drop
input supplied in the first line of the file.
A fallback decoder is used when the magic in the first line of the file
does not match any of the other known decoders. In such a case,
the fallback decoder is constructed and provided a LineBufferedReader
instance. The process of matching magic only peeks the first non-empty
line, so it is available for re-reading in Decode() using ReadLine().
There can be only one fallback decoder per type; a second attempt of
registering a fallback will result in an exception to avoid bugs.
To address the issue of parsing failing on badly or non-headered files,
set the legacy decoders for Beatmaps and Storyboards as the fallbacks.
Due to non-trivial logic, several new, passing unit tests with possible
edge cases also included.
2019-09-15 01:28:07 +02:00
Bartłomiej Dach
7b1ff38df7
Implement line-buffered reader
...
Add a line-buffered reader decorator operating on StreamReader
instances. The decorator has two main operations - PeekLine(), which
allows to see the next line in the stream without consuming it,
ReadLine(), which consumes and returns the next line in the stream, and
ReadToEnd() which reads all the remaining text in the stream (including
the unconsumed peeked line). Peeking line-per-line uses an internal
queue of lines that have been read ahead from the underlying stream.
The addition of the line-buffered reader is a workaround solution to
a problem with decoding. At current selecting a decoder works by
irreversibly reading the first line from the stream and looking for
a magic string that indicates the type of decoder to use.
It might however be possible for a file to be valid in format, just
missing a header. In such a case a lack of a line-buffered reader makes
it impossible to reparse the content of that first line. Introducing it
will however allow to peek the first line for magic first.
- If magic is found in the first line, GetDecoder() will peek it and
use it to return the correct Decoder instance. Note that in the case
of JsonBeatmapDecoder the magic is the opening JSON object brace,
and therefore must not be consumed.
- If magic is not found, the fallback decoder will be able to consume
it using ReadLine() in Decode().
This commit additionally contains basic unit tests for the reader.
Suggested-by: Aergwyn <aergwyn@t-online.de>
2019-09-15 01:26:15 +02:00
Dean Herbert
5b75060b94
Add test for rollback logic correctly dereferencing files
2019-06-10 16:45:45 +09:00
Dean Herbert
80d65f9a3b
Update resource stores with GetAvailableResources
2019-05-31 14:33:18 +09:00
Dean Herbert
612db31c38
Apply newline additions
2019-04-01 12:16:32 +09:00
Dean Herbert
26d53d06a9
Fix remaining issues
2019-02-28 13:31:40 +09:00
Dean Herbert
a580c17949
Fix codefactor issue
2019-01-24 17:46:09 +09:00
Dean Herbert
8617aaa2a7
Update licence header (and remove year)
2019-01-24 17:43:03 +09:00
smoogipoo
a783fdb501
Add LegacyFileArchiveReader
...
Treats files as "archives" for import.
2018-11-28 16:13:16 +09:00
smoogipoo
0982508d26
Rename to LegacyDirectoryArchiveReader
2018-11-28 15:13:27 +09:00
smoogipoo
14ff4bc26a
Move primary key attribute to IHasPrimaryKey
2018-11-28 13:19:23 +09:00
smoogipoo
f27bd3ef3e
OpenTK -> osuTK
2018-11-20 17:14:59 +09:00
smoogipoo
21d5322899
Update with async changes
2018-08-27 17:05:58 +09:00
Dean Herbert
c756a89daf
Standardise path
2018-06-06 20:55:42 +09:00
Dean Herbert
3b03a25ea5
Fix beatmaps with subfolders importing incorrectly on windows
...
Closes #2718 .
2018-06-05 11:36:44 +09:00
smoogipoo
397d93660a
Don't deep-clone beatmapinfo/control points
2018-05-17 13:17:33 +09:00
Dean Herbert
44a9aa529a
Test CI run with new changes
2018-04-22 03:51:17 +09:00
Dean Herbert
32a74f95a5
Normalize all the line endings
2018-04-13 18:26:38 +09:00
smoogipoo
c29f7a4333
Merge remote-tracking branch 'origin/master' into netstandard
2018-03-24 14:49:46 +09:00
Dean Herbert
b9ef32b09b
Further xmldoc and restoring of Cleanup method
2018-02-15 16:31:42 +09:00
Dean Herbert
1b13be1372
Cleanups and xmldoc additions
2018-02-15 16:24:14 +09:00
Dean Herbert
d3dd31dadb
Make deletion and purging logic even more global
2018-02-15 14:22:39 +09:00
Dean Herbert
d340509b1d
Move ArchiveReaders to a more global namespace
...
Also moves delete and action logic to a shared implementation
2018-02-15 14:22:39 +09:00
Dean Herbert
6ff63c2f0c
Move deletion to ArchiveModelImportManager
2018-02-15 14:22:39 +09:00
Dean Herbert
e0d28564d0
Move import logic to shared implementation
2018-02-15 14:22:39 +09:00
Dean Herbert
a738664167
Add interface for database context factory
2018-02-12 23:10:05 +09:00
Dean Herbert
8b37fde15b
Only write when writes occur
...
Also add finaliser logic for safety. Also better threading. Also more cleanup.
2018-02-12 19:57:21 +09:00
Dean Herbert
edc3638175
DatabaseWriteUsage
2018-02-12 17:55:11 +09:00
smoogipoo
b500b76407
Merge branch 'master' into update-branch
2018-02-08 23:35:48 +09:00
Dean Herbert
37d393bca0
Update licence headers
2018-01-05 20:21:19 +09:00
smoogipoo
af86220dcc
Merge remote-tracking branch 'origin/master' into update-branch
2018-01-05 12:08:50 +09:00
smoogipoo
5eb2dc3d64
Cleanup some possible CI errors
...
Not sure why these aren't being detected on master right now - they should be...
2018-01-05 01:20:24 +09:00
smoogipoo
7a793eaded
Merge master into netstandard
2017-12-27 20:48:53 +09:00
smoogipoo
bdf283a4e1
A bit more cleanup
2017-12-07 15:55:38 +09:00
smoogipoo
245b5f759f
Underscore + lowercase all keys
2017-12-07 14:37:12 +09:00
smoogipoo
d026587a91
Add flag to explicitly serialize the type version
2017-12-07 12:49:16 +09:00
smoogipoo
9e51480aa3
Cleanup TypedListConverter a bit
2017-12-07 12:27:30 +09:00
smoogipoo
feef4b1890
Add license header
2017-12-07 12:18:00 +09:00
smoogipoo
851c20aff0
Add a few comments
2017-12-07 11:17:32 +09:00
smoogipoo
9787788081
Revert unintended change
2017-12-07 03:39:43 +09:00
smoogipoo
4232a54b32
Make TypedListConverter not reconstruct serializers
2017-12-07 03:04:56 +09:00
smoogipoo
b6b26cfe25
Add basic method to migrate beatmaps to the new JSON format
2017-12-07 03:04:36 +09:00
