1
0
mirror of https://github.com/ppy/osu.git synced 2024-12-27 05:03:06 +08:00
Commit Graph

96 Commits

Author SHA1 Message Date
Dean Herbert
6268bbcfc8
Merge branch 'master' into beatmap-parsing-fallback-v2 2019-10-03 15:12:21 +08:00
Dean Herbert
d0a4e1e3c2 Force a checksum check before skipping FileStore copy op 2019-09-20 15:00:27 +09:00
Bartłomiej Dach
86588778b1 Implement fallback decoder registration
After the preparatory introduction of LineBufferedReader, it is now
possible to introduce registration of fallback decoders that won't drop
input supplied in the first line of the file.

A fallback decoder is used when the magic in the first line of the file
does not match any of the other known decoders. In such a case,
the fallback decoder is constructed and provided a LineBufferedReader
instance. The process of matching magic only peeks the first non-empty
line, so it is available for re-reading in Decode() using ReadLine().

There can be only one fallback decoder per type; a second attempt of
registering a fallback will result in an exception to avoid bugs.

To address the issue of parsing failing on badly or non-headered files,
set the legacy decoders for Beatmaps and Storyboards as the fallbacks.

Due to non-trivial logic, several new, passing unit tests with possible
edge cases also included.
2019-09-15 01:28:07 +02:00
Bartłomiej Dach
7b1ff38df7 Implement line-buffered reader
Add a line-buffered reader decorator operating on StreamReader
instances. The decorator has two main operations - PeekLine(), which
allows to see the next line in the stream without consuming it,
ReadLine(), which consumes and returns the next line in the stream, and
ReadToEnd() which reads all the remaining text in the stream (including
the unconsumed peeked line). Peeking line-per-line uses an internal
queue of lines that have been read ahead from the underlying stream.

The addition of the line-buffered reader is a workaround solution to
a problem with decoding. At current selecting a decoder works by
irreversibly reading the first line from the stream and looking for
a magic string that indicates the type of decoder to use.

It might however be possible for a file to be valid in format, just
missing a header. In such a case a lack of a line-buffered reader makes
it impossible to reparse the content of that first line. Introducing it
will however allow to peek the first line for magic first.

 - If magic is found in the first line, GetDecoder() will peek it and
   use it to return the correct Decoder instance. Note that in the case
   of JsonBeatmapDecoder the magic is the opening JSON object brace,
   and therefore must not be consumed.

 - If magic is not found, the fallback decoder will be able to consume
   it using ReadLine() in Decode().

This commit additionally contains basic unit tests for the reader.

Suggested-by: Aergwyn <aergwyn@t-online.de>
2019-09-15 01:26:15 +02:00
Dean Herbert
5b75060b94 Add test for rollback logic correctly dereferencing files 2019-06-10 16:45:45 +09:00
Dean Herbert
80d65f9a3b Update resource stores with GetAvailableResources 2019-05-31 14:33:18 +09:00
Dean Herbert
612db31c38 Apply newline additions 2019-04-01 12:16:32 +09:00
Dean Herbert
26d53d06a9 Fix remaining issues 2019-02-28 13:31:40 +09:00
Dean Herbert
a580c17949 Fix codefactor issue 2019-01-24 17:46:09 +09:00
Dean Herbert
8617aaa2a7 Update licence header (and remove year) 2019-01-24 17:43:03 +09:00
smoogipoo
a783fdb501 Add LegacyFileArchiveReader
Treats files as "archives" for import.
2018-11-28 16:13:16 +09:00
smoogipoo
0982508d26 Rename to LegacyDirectoryArchiveReader 2018-11-28 15:13:27 +09:00
smoogipoo
14ff4bc26a Move primary key attribute to IHasPrimaryKey 2018-11-28 13:19:23 +09:00
smoogipoo
f27bd3ef3e OpenTK -> osuTK 2018-11-20 17:14:59 +09:00
smoogipoo
21d5322899 Update with async changes 2018-08-27 17:05:58 +09:00
Dean Herbert
c756a89daf Standardise path 2018-06-06 20:55:42 +09:00
Dean Herbert
3b03a25ea5 Fix beatmaps with subfolders importing incorrectly on windows
Closes #2718.
2018-06-05 11:36:44 +09:00
smoogipoo
397d93660a Don't deep-clone beatmapinfo/control points 2018-05-17 13:17:33 +09:00
Dean Herbert
44a9aa529a Test CI run with new changes 2018-04-22 03:51:17 +09:00
Dean Herbert
32a74f95a5 Normalize all the line endings 2018-04-13 18:26:38 +09:00
smoogipoo
c29f7a4333 Merge remote-tracking branch 'origin/master' into netstandard 2018-03-24 14:49:46 +09:00
Dean Herbert
b9ef32b09b Further xmldoc and restoring of Cleanup method 2018-02-15 16:31:42 +09:00
Dean Herbert
1b13be1372 Cleanups and xmldoc additions 2018-02-15 16:24:14 +09:00
Dean Herbert
d3dd31dadb Make deletion and purging logic even more global 2018-02-15 14:22:39 +09:00
Dean Herbert
d340509b1d Move ArchiveReaders to a more global namespace
Also moves delete and action logic to a shared implementation
2018-02-15 14:22:39 +09:00
Dean Herbert
6ff63c2f0c Move deletion to ArchiveModelImportManager 2018-02-15 14:22:39 +09:00
Dean Herbert
e0d28564d0 Move import logic to shared implementation 2018-02-15 14:22:39 +09:00
Dean Herbert
a738664167 Add interface for database context factory 2018-02-12 23:10:05 +09:00
Dean Herbert
8b37fde15b Only write when writes occur
Also add finaliser logic for safety. Also better threading. Also more cleanup.
2018-02-12 19:57:21 +09:00
Dean Herbert
edc3638175 DatabaseWriteUsage 2018-02-12 17:55:11 +09:00
smoogipoo
b500b76407 Merge branch 'master' into update-branch 2018-02-08 23:35:48 +09:00
Dean Herbert
37d393bca0 Update licence headers 2018-01-05 20:21:19 +09:00
smoogipoo
af86220dcc Merge remote-tracking branch 'origin/master' into update-branch 2018-01-05 12:08:50 +09:00
smoogipoo
5eb2dc3d64 Cleanup some possible CI errors
Not sure why these aren't being detected on master right now - they should be...
2018-01-05 01:20:24 +09:00
smoogipoo
7a793eaded Merge master into netstandard 2017-12-27 20:48:53 +09:00
smoogipoo
bdf283a4e1 A bit more cleanup 2017-12-07 15:55:38 +09:00
smoogipoo
245b5f759f Underscore + lowercase all keys 2017-12-07 14:37:12 +09:00
smoogipoo
d026587a91 Add flag to explicitly serialize the type version 2017-12-07 12:49:16 +09:00
smoogipoo
9e51480aa3 Cleanup TypedListConverter a bit 2017-12-07 12:27:30 +09:00
smoogipoo
feef4b1890 Add license header 2017-12-07 12:18:00 +09:00
smoogipoo
851c20aff0 Add a few comments 2017-12-07 11:17:32 +09:00
smoogipoo
9787788081 Revert unintended change 2017-12-07 03:39:43 +09:00
smoogipoo
4232a54b32 Make TypedListConverter not reconstruct serializers 2017-12-07 03:04:56 +09:00
smoogipoo
b6b26cfe25 Add basic method to migrate beatmaps to the new JSON format 2017-12-07 03:04:36 +09:00
smoogipoo
e199ee5225 Add a few xmldocs 2017-12-06 00:42:17 +09:00
smoogipoo
b584178e85 Make Beatmap ISerializable and add more JsonIgnores 2017-12-06 00:37:37 +09:00
smoogipoo
aac41d2de6 Disable resharper inspections on case-by-case basis 2017-11-20 18:55:48 +09:00
smoogipoo
d2ab0621f3 Make osu.Game + rulesets compile with netstandard 2017-11-19 13:46:51 +09:00
smoogipoo
04b271cce9 Migrate osu.Game project to VS2017 project structure 2017-10-30 22:24:11 +09:00
TocoToucan
5107489cda Remove DatabaseBackedStore Reset functions 2017-10-25 15:12:14 +03:00