1
0
mirror of https://github.com/ppy/osu.git synced 2024-12-18 23:02:55 +08:00
Commit Graph

150 Commits

Author SHA1 Message Date
Dean Herbert
6268bbcfc8
Merge branch 'master' into beatmap-parsing-fallback-v2 2019-10-03 15:12:21 +08:00
Dean Herbert
a06cb54732
Merge branch 'master' into beatmap-parsing-fallback-v2 2019-09-20 15:28:08 +09:00
Dean Herbert
f306fe27d8 Add test to cover corruption case 2019-09-20 15:05:48 +09:00
Dean Herbert
ddff9882cf Fix importing archives which are nested in a single folder within a zip 2019-09-19 19:11:04 +09:00
Bartłomiej Dach
86588778b1 Implement fallback decoder registration
After the preparatory introduction of LineBufferedReader, it is now
possible to introduce registration of fallback decoders that won't drop
input supplied in the first line of the file.

A fallback decoder is used when the magic in the first line of the file
does not match any of the other known decoders. In such a case,
the fallback decoder is constructed and provided a LineBufferedReader
instance. The process of matching magic only peeks the first non-empty
line, so it is available for re-reading in Decode() using ReadLine().

There can be only one fallback decoder per type; a second attempt of
registering a fallback will result in an exception to avoid bugs.

To address the issue of parsing failing on badly or non-headered files,
set the legacy decoders for Beatmaps and Storyboards as the fallbacks.

Due to non-trivial logic, several new, passing unit tests with possible
edge cases also included.
2019-09-15 01:28:07 +02:00
Bartłomiej Dach
11eda44d34 Migrate decoding to line-buffered reader
Migrate all usages of StreamReader in the context of decoding beatmaps,
storyboards or skins to the new LineBufferedReader.
2019-09-15 01:28:07 +02:00
Bartłomiej Dach
7b1ff38df7 Implement line-buffered reader
Add a line-buffered reader decorator operating on StreamReader
instances. The decorator has two main operations - PeekLine(), which
allows to see the next line in the stream without consuming it,
ReadLine(), which consumes and returns the next line in the stream, and
ReadToEnd() which reads all the remaining text in the stream (including
the unconsumed peeked line). Peeking line-per-line uses an internal
queue of lines that have been read ahead from the underlying stream.

The addition of the line-buffered reader is a workaround solution to
a problem with decoding. At current selecting a decoder works by
irreversibly reading the first line from the stream and looking for
a magic string that indicates the type of decoder to use.

It might however be possible for a file to be valid in format, just
missing a header. In such a case a lack of a line-buffered reader makes
it impossible to reparse the content of that first line. Introducing it
will however allow to peek the first line for magic first.

 - If magic is found in the first line, GetDecoder() will peek it and
   use it to return the correct Decoder instance. Note that in the case
   of JsonBeatmapDecoder the magic is the opening JSON object brace,
   and therefore must not be consumed.

 - If magic is not found, the fallback decoder will be able to consume
   it using ReadLine() in Decode().

This commit additionally contains basic unit tests for the reader.

Suggested-by: Aergwyn <aergwyn@t-online.de>
2019-09-15 01:26:15 +02:00
Dean Herbert
9e1cb90dd8 Remove existing argument for ItemAdded event
For all usages, it looks like this was unnecessary.
2019-06-26 11:40:41 +09:00
Dean Herbert
12aa264657 Consolidate tests and check for file reference counts 2019-06-10 18:35:23 +09:00
Dean Herbert
28b2a516e3 Ensure exception is only thrown once on rollback 2019-06-10 18:13:33 +09:00
Dean Herbert
dcdb806120 Catch newly thrown exception in test 2019-06-10 17:26:56 +09:00
Dean Herbert
5b75060b94 Add test for rollback logic correctly dereferencing files 2019-06-10 16:45:45 +09:00
smoogipoo
f090e292c9 Move ArchiveModelManager import process to async flow 2019-06-10 13:42:22 +09:00
Dean Herbert
c39c37a18d Apply more missed cases 2019-04-01 12:44:46 +09:00
smoogipoo
00191ca940 Actually set the beatmap difficulty 2019-03-12 15:24:35 +09:00
smoogipoo
059397ac50 Remove unnecessary early return for maching beatmap IDs 2019-03-12 14:40:13 +09:00
Dean Herbert
d0ae75af6e Add tests and fix scenario where all matching are contained by duplicate candidate 2019-03-11 18:13:33 +09:00
Dean Herbert
80737b9ef8 Remove "silent" parameter; consolidate import logic 2019-02-25 18:24:06 +09:00
Dean Herbert
628b3e1119 Move osu-resources to nuget 2019-01-28 18:33:39 +09:00
Dean Herbert
8617aaa2a7 Update licence header (and remove year) 2019-01-24 17:43:03 +09:00
Dean Herbert
c45c34d400 Make beatmap importing possible elsewhere in tests 2018-12-25 19:17:21 +09:00
smoogipoo
a8ad7d4670 Add silent import parameter 2018-11-30 18:27:13 +09:00
Dean Herbert
10095d09c3 Fix test namespace 2018-11-30 17:02:01 +09:00
Dean Herbert
3727955911 Refactors/renames 2018-11-30 16:35:45 +09:00
Dean Herbert
831cd3ed59 ScoreInfo scoreInfo -> ScoreInfo score 2018-11-30 15:18:52 +09:00
smoogipoo
2bedffc00f Add tests 2018-11-30 14:27:34 +09:00
smoogipoo
a88b69ec43 Notify of existing imports 2018-11-28 20:19:35 +09:00
smoogipoo
6c100caba7 Use existing instance of beatmap rather than adding more resources 2018-06-17 19:00:58 +09:00
Dean Herbert
955a78e76d Remove OnlineBeatmapSetID from BeatmapInfo
It should now be retrieved via the linked BeatmapSetInfo
2018-06-08 15:26:27 +09:00
Dean Herbert
8220e0c79a Remove OnlineBeatmapSetID from BeatmapMetadata 2018-06-08 12:46:34 +09:00
smoogipoo
3745f9000b Merge remote-tracking branch 'origin/master' into remove-decoder-offset 2018-06-07 13:37:19 +09:00
ekrctb
aeeb03ff9c simpler temporary path generation 2018-06-06 18:36:43 +09:00
ekrctb
b84441ab87 Fix TestImportOverIPC 2018-06-04 22:25:18 +09:00
Dean Herbert
203691b1c7 Add import rollback test 2018-05-29 16:23:38 +09:00
Dean Herbert
cc081cad5a Simplify test osz instantiation 2018-05-29 16:23:38 +09:00
Dean Herbert
015fd9d0e7 Fix loading test method returning oldest import rather than newest 2018-05-29 16:23:38 +09:00
smoogipoo
b267ec3178 Fix failing test 2018-05-28 03:48:09 +09:00
smoogipoo
66b3b295e7 Use IBeatmap wherever possible 2018-04-19 20:44:38 +09:00
Dean Herbert
32a74f95a5 Normalize all the line endings 2018-04-13 18:26:38 +09:00
smoogipoo
c21e820a45 One more level of nesting (we can do this better in the future) 2018-03-25 02:43:45 +09:00
Dean Herbert
217dd2ecdc Initial push for better decoders 2018-03-09 21:24:15 +09:00
Dean Herbert
e46f363fdc Fix failing unit test 2018-03-04 22:46:50 +09:00
Dean Herbert
d340509b1d Move ArchiveReaders to a more global namespace
Also moves delete and action logic to a shared implementation
2018-02-15 14:22:39 +09:00
Dean Herbert
e0d28564d0 Move import logic to shared implementation 2018-02-15 14:22:39 +09:00
Dean Herbert
9ed05543d7 Fix post-test conditionals from being inverse of what we want to test 2018-02-12 15:39:00 +09:00
Dean Herbert
264a0f59e2 Fix duplicate test name 2018-02-12 12:22:13 +09:00
Dean Herbert
721bb7e4dd Add proper handling for OnlineBeatmapSetID conflicts
Not yet working
2018-02-09 21:31:33 +09:00
Dean Herbert
541068235d Test import twice in a row 2018-02-09 18:23:06 +09:00
Dean Herbert
623ba652ed Share more code between tests 2018-02-09 18:20:18 +09:00
Dean Herbert
981fa379b7 Count() -> Count 2018-02-09 18:02:28 +09:00