h1or4dme8/osu-lazer
1
0
Fork 0
mirror of https://github.com/ppy/osu.git synced 2024-11-06 14:37:25 +08:00
Code Issues Releases Wiki Activity
46,582 Commits 3 Branches 653 Tags 366 MiB
0633f3bcfe
Commit Graph

201 Commits

Author SHA1 Message Date
Shivam
c3cd2a74f5 Move general purpose migration to MigratableStorage 2020-07-01 22:57:16 +02:00
smoogipoo
cdcad94e9f Handle exception thrown due to custom stoage on startup 2020-07-01 17:47:29 +09:00
smoogipoo
ab15b6031d Update with framework-side storage changes 2020-07-01 17:12:07 +09:00
Shivam
47a732ef60 Address review comments 
Now asserting instead of an if-statement, change cast from OsuStorage to MigratableStorage and make internal virtual properties protected.
 2020-06-24 23:01:56 +02:00
Shivam
af11340849 Fix nullref exceptions and redundant explicit type 2020-06-24 02:13:28 +02:00
Shivam
c32ef5e718 Address formatting issues 2020-06-24 00:37:29 +02:00
Shivam
8e8458ab8f make migrate public abstract in base and override 2020-06-23 23:58:28 +02:00
Shivam
a47d34f1db make ignore properties protected virtual get-only in base 2020-06-23 23:34:26 +02:00
Shivam
a899c754f1 Remove whitespace at the end of xmldoc line 2020-06-22 13:03:24 +02:00
Shivam
a94dcc4923 Add xmldoc to MigratableStorage 2020-06-22 12:59:38 +02:00
Shivam
6b14079c0a InspectCode changes 2020-06-22 12:43:01 +02:00
Shivam
f878388d57 Fix TestMigrationToSeeminglyNestedTarget failing 2020-06-22 11:56:14 +02:00
Shivam
21774b8967 Move static properties to parent class and inherit OsuStorage from it 2020-06-22 11:38:50 +02:00
Shivam
dd9697032c Introduce new class MigratableStorage 2020-06-16 17:39:20 +02:00
smoogipoo
092f5b6521 Fix incorrect reference + simplify 2020-06-03 17:41:05 +09:00
smoogipoo
8aa8d2c880 Resolve NREs 2020-06-03 16:59:37 +09:00
Dean Herbert
6904d5d247 Remove unnecessary override 2020-05-25 13:12:53 +09:00
Dean Herbert
1977affe7e Fix OpenInNativeExplorer not working correctly for wrapped storages 2020-05-25 09:27:11 +09:00
Dean Herbert
6ca102bc3f Attempt delete operations more than once 2020-05-15 13:19:03 +09:00
Dean Herbert
9ed0e8891b Merge branch 'fix-nested-folder-migration-endless-copy' into migration-ui 2020-05-15 13:04:16 +09:00
Dean Herbert
0690d81bbb Add protection against migrating to a nested folder 2020-05-15 10:40:26 +09:00
Dean Herbert
25bbb02999 Throw better exceptions from OsuStorage 2020-05-15 10:28:03 +09:00
Dean Herbert
364aa5aa12 Add protection against migrating to a nested folder 2020-05-14 22:45:03 +09:00
Dean Herbert
827d75b152 Revert "Add protection against migrating to a nested folder" 
This reverts commit ef8375b442.
 2020-05-14 22:44:27 +09:00
Dean Herbert
ef8375b442 Add protection against migrating to a nested folder 2020-05-14 22:42:42 +09:00
Dean Herbert
06f507496a Delete migration source if no files exist after completion 2020-05-14 17:41:55 +09:00
Dean Herbert
cb0b25ac55 Throw better exceptions from OsuStorage 2020-05-14 17:41:55 +09:00
Dean Herbert
ad1d050fb4 Throw exception on copy timeout 2020-05-13 20:29:15 +09:00
Dean Herbert
49e616b7e5 Also check for directory presence before migrating 2020-05-13 20:19:14 +09:00
Dean Herbert
9de216f1b4 Merge branch 'master' into migration-backend 2020-05-13 20:15:22 +09:00
Huo Yaoyuan
f07d95ac59 Use IReadOnlyList<T> for TypedListConverter. 2020-05-12 21:49:55 +08:00
Huo Yaoyuan
f7d26688f4 Merge branch 'master' into json-converter 2020-05-12 21:43:02 +08:00
Dean Herbert
e4a23b3e7d Fix ignored excluding more than top level 2020-05-12 12:39:04 +09:00
Dean Herbert
984f27c107 Add simple retry logic on file copy failure (may be in use) 2020-05-11 21:39:44 +09:00
Dean Herbert
a11be07bb1 Move log storage location after migration complete 2020-05-11 21:39:44 +09:00
Dean Herbert
36640f5dda Merge branch 'master' into migration-backend 2020-05-11 18:52:36 +09:00
Dean Herbert
384862d48b Fix incorrect relative paths when using GetStorageForDirectory 2020-05-10 13:20:13 +09:00
Dean Herbert
bbebd26efb Use DirectoryInfo in more places 2020-05-09 20:13:31 +09:00
Huo Yaoyuan
80a193a616 Use IEnumerable<T> for TypedListConverter. 2020-05-09 17:41:59 +08:00
Huo Yaoyuan
74cbe9306c Use strongly-typed JsonConerter. 2020-05-09 16:39:46 +08:00
Dean Herbert
7a2020fd45 User copy operation instead of move 2020-05-07 22:12:24 +09:00
Dean Herbert
49a03f1c06 Add basic blocking migration, move not copy 2020-05-07 22:11:57 +09:00
Dean Herbert
7781408643 Update in line with framework storage changes 2020-05-06 18:28:24 +09:00
smoogipoo
d27ca725f9 Use IEnumerable<T> instead 2020-04-22 09:04:44 +09:00
smoogipoo
a702a521f8 Fix not being able to serialise converted beatmaps 2020-04-22 09:04:44 +09:00
smoogipoo
f6f5de7ad1 Allow LineBufferedReader to keep underlying stream open 2020-03-31 10:13:50 +09:00
Dean Herbert
022465f546 Add encoding and import support 2020-03-24 14:51:52 +09:00
Dean Herbert
68ebe98fde Remove unused GetUnderlyingStream method 2020-03-24 14:08:25 +09:00
Huo Yaoyuan
b14c7bd171 Use type switch in SearializationWriter. 2020-01-27 21:49:34 +08:00
Huo Yaoyuan
40b43b85f1 CA1715: use prefix for generic parameters. 2019-12-17 13:00:25 +08:00
Huo Yaoyuan
e46f6627e4 CA1052: make type static. 2019-12-17 13:00:13 +08:00
Huo Yaoyuan
31cc0d13da Use 'out var'. 2019-11-12 19:55:26 +08:00
Dean Herbert
35be8f9dfb Share framework file-exclusion function 2019-10-30 19:34:17 +09:00
Bartłomiej Dach
4b84564f47
Switch casing comparison mode to ordinal 
Switch from InvariantCultureIgnoreCase to OrdinalIgnoreCase when
checking file paths in archives for substrings indicating the file can
be ignored for performance gains.

Co-Authored-By: Dan Balasescu <smoogipoo@smgi.me>
 2019-10-11 09:24:41 +02:00
Bartłomiej Dach
57bfa18359 Filter out OS-generated files from archives 
Add a filename ignore list to ZipArchiveReader to filter out superfluous
OS-generated files from archives during the import process. In addition
to decreasing the size of files imported this allows imports of some
incorrectly-constructed archives. An example is the case of having
a __MACOSX directory next to a single directory with the actual files -
filtering out the former at ZipArchiveReader allows the fallback added
in #6170 to work.
 2019-10-10 23:54:23 +02:00
Dean Herbert
6268bbcfc8
Merge branch 'master' into beatmap-parsing-fallback-v2 2019-10-03 15:12:21 +08:00
Dean Herbert
d0a4e1e3c2 Force a checksum check before skipping FileStore copy op 2019-09-20 15:00:27 +09:00
Bartłomiej Dach
86588778b1 Implement fallback decoder registration 
After the preparatory introduction of LineBufferedReader, it is now
possible to introduce registration of fallback decoders that won't drop
input supplied in the first line of the file.

A fallback decoder is used when the magic in the first line of the file
does not match any of the other known decoders. In such a case,
the fallback decoder is constructed and provided a LineBufferedReader
instance. The process of matching magic only peeks the first non-empty
line, so it is available for re-reading in Decode() using ReadLine().

There can be only one fallback decoder per type; a second attempt of
registering a fallback will result in an exception to avoid bugs.

To address the issue of parsing failing on badly or non-headered files,
set the legacy decoders for Beatmaps and Storyboards as the fallbacks.

Due to non-trivial logic, several new, passing unit tests with possible
edge cases also included.
 2019-09-15 01:28:07 +02:00
Bartłomiej Dach
7b1ff38df7 Implement line-buffered reader 
Add a line-buffered reader decorator operating on StreamReader
instances. The decorator has two main operations - PeekLine(), which
allows to see the next line in the stream without consuming it,
ReadLine(), which consumes and returns the next line in the stream, and
ReadToEnd() which reads all the remaining text in the stream (including
the unconsumed peeked line). Peeking line-per-line uses an internal
queue of lines that have been read ahead from the underlying stream.

The addition of the line-buffered reader is a workaround solution to
a problem with decoding. At current selecting a decoder works by
irreversibly reading the first line from the stream and looking for
a magic string that indicates the type of decoder to use.

It might however be possible for a file to be valid in format, just
missing a header. In such a case a lack of a line-buffered reader makes
it impossible to reparse the content of that first line. Introducing it
will however allow to peek the first line for magic first.

 - If magic is found in the first line, GetDecoder() will peek it and
   use it to return the correct Decoder instance. Note that in the case
   of JsonBeatmapDecoder the magic is the opening JSON object brace,
   and therefore must not be consumed.

 - If magic is not found, the fallback decoder will be able to consume
   it using ReadLine() in Decode().

This commit additionally contains basic unit tests for the reader.

Suggested-by: Aergwyn <aergwyn@t-online.de>
 2019-09-15 01:26:15 +02:00
Dean Herbert
5b75060b94 Add test for rollback logic correctly dereferencing files 2019-06-10 16:45:45 +09:00
Dean Herbert
80d65f9a3b Update resource stores with GetAvailableResources 2019-05-31 14:33:18 +09:00
Dean Herbert
612db31c38 Apply newline additions 2019-04-01 12:16:32 +09:00
Dean Herbert
26d53d06a9 Fix remaining issues 2019-02-28 13:31:40 +09:00
Dean Herbert
a580c17949 Fix codefactor issue 2019-01-24 17:46:09 +09:00
Dean Herbert
8617aaa2a7 Update licence header (and remove year) 2019-01-24 17:43:03 +09:00
smoogipoo
a783fdb501 Add LegacyFileArchiveReader 
Treats files as "archives" for import.
 2018-11-28 16:13:16 +09:00
smoogipoo
0982508d26 Rename to LegacyDirectoryArchiveReader 2018-11-28 15:13:27 +09:00
smoogipoo
14ff4bc26a Move primary key attribute to IHasPrimaryKey 2018-11-28 13:19:23 +09:00
smoogipoo
f27bd3ef3e OpenTK -> osuTK 2018-11-20 17:14:59 +09:00
smoogipoo
21d5322899 Update with async changes 2018-08-27 17:05:58 +09:00
Dean Herbert
c756a89daf Standardise path 2018-06-06 20:55:42 +09:00
Dean Herbert
3b03a25ea5 Fix beatmaps with subfolders importing incorrectly on windows 
Closes #2718.
 2018-06-05 11:36:44 +09:00
smoogipoo
397d93660a Don't deep-clone beatmapinfo/control points 2018-05-17 13:17:33 +09:00
Dean Herbert
44a9aa529a Test CI run with new changes 2018-04-22 03:51:17 +09:00
Dean Herbert
32a74f95a5 Normalize all the line endings 2018-04-13 18:26:38 +09:00
smoogipoo
c29f7a4333 Merge remote-tracking branch 'origin/master' into netstandard 2018-03-24 14:49:46 +09:00
Dean Herbert
b9ef32b09b Further xmldoc and restoring of Cleanup method 2018-02-15 16:31:42 +09:00
Dean Herbert
1b13be1372 Cleanups and xmldoc additions 2018-02-15 16:24:14 +09:00
Dean Herbert
d3dd31dadb Make deletion and purging logic even more global 2018-02-15 14:22:39 +09:00
Dean Herbert
d340509b1d Move ArchiveReaders to a more global namespace 
Also moves delete and action logic to a shared implementation
 2018-02-15 14:22:39 +09:00
Dean Herbert
6ff63c2f0c Move deletion to ArchiveModelImportManager 2018-02-15 14:22:39 +09:00
Dean Herbert
e0d28564d0 Move import logic to shared implementation 2018-02-15 14:22:39 +09:00
Dean Herbert
a738664167 Add interface for database context factory 2018-02-12 23:10:05 +09:00
Dean Herbert
8b37fde15b Only write when writes occur 
Also add finaliser logic for safety. Also better threading. Also more cleanup.
 2018-02-12 19:57:21 +09:00
Dean Herbert
edc3638175 DatabaseWriteUsage 2018-02-12 17:55:11 +09:00
smoogipoo
b500b76407 Merge branch 'master' into update-branch 2018-02-08 23:35:48 +09:00
Dean Herbert
37d393bca0 Update licence headers 2018-01-05 20:21:19 +09:00
smoogipoo
af86220dcc Merge remote-tracking branch 'origin/master' into update-branch 2018-01-05 12:08:50 +09:00
smoogipoo
5eb2dc3d64 Cleanup some possible CI errors 
Not sure why these aren't being detected on master right now - they should be...
 2018-01-05 01:20:24 +09:00
smoogipoo
7a793eaded Merge master into netstandard 2017-12-27 20:48:53 +09:00
smoogipoo
bdf283a4e1 A bit more cleanup 2017-12-07 15:55:38 +09:00
smoogipoo
245b5f759f Underscore + lowercase all keys 2017-12-07 14:37:12 +09:00
smoogipoo
d026587a91 Add flag to explicitly serialize the type version 2017-12-07 12:49:16 +09:00
smoogipoo
9e51480aa3 Cleanup TypedListConverter a bit 2017-12-07 12:27:30 +09:00
smoogipoo
feef4b1890 Add license header 2017-12-07 12:18:00 +09:00
smoogipoo
851c20aff0 Add a few comments 2017-12-07 11:17:32 +09:00
smoogipoo
9787788081 Revert unintended change 2017-12-07 03:39:43 +09:00
smoogipoo
4232a54b32 Make TypedListConverter not reconstruct serializers 2017-12-07 03:04:56 +09:00
smoogipoo
b6b26cfe25 Add basic method to migrate beatmaps to the new JSON format 2017-12-07 03:04:36 +09:00
smoogipoo
e199ee5225 Add a few xmldocs 2017-12-06 00:42:17 +09:00