208 Commits

Author	SHA1	Message	Date
Dean Herbert	1989141968	Merge branch 'master' into tourney-asset-refactor	2020-10-07 21:34:07 +09:00
Dean Herbert	58e84760b9	Fix path empty string check causing regression in behaviour	2020-09-03 19:17:07 +09:00
Dean Herbert	218cc39a4c	Avoid throwing exceptions when MutatePath is called with null path	2020-09-03 13:20:23 +09:00
Shivam	c167727ac6	Merge branch 'master' into tourney-asset-refactor ... # Conflicts: # osu.Game/IO/OsuStorage.cs	2020-08-09 12:38:41 +02:00
smoogipoo	00a2fbce06	Fix test failures	2020-07-06 22:41:58 +09:00
smoogipoo	1effe71ec2	Add dialog for storage options	2020-07-06 22:03:09 +09:00
Shivam	66e61aacff	Logger now shows the actual path of the destination ... Forgot to change this while changing the param from string to Storage	2020-07-02 00:32:09 +02:00
Shivam	c3cd2a74f5	Move general purpose migration to MigratableStorage	2020-07-01 22:57:16 +02:00
smoogipoo	cdcad94e9f	Handle exception thrown due to custom stoage on startup	2020-07-01 17:47:29 +09:00
smoogipoo	ab15b6031d	Update with framework-side storage changes	2020-07-01 17:12:07 +09:00
Shivam	47a732ef60	Address review comments ... Now asserting instead of an if-statement, change cast from OsuStorage to MigratableStorage and make internal virtual properties protected.	2020-06-24 23:01:56 +02:00
Shivam	af11340849	Fix nullref exceptions and redundant explicit type	2020-06-24 02:13:28 +02:00
Shivam	c32ef5e718	Address formatting issues	2020-06-24 00:37:29 +02:00
Shivam	8e8458ab8f	make migrate public abstract in base and override	2020-06-23 23:58:28 +02:00
Shivam	a47d34f1db	make ignore properties protected virtual get-only in base	2020-06-23 23:34:26 +02:00
Shivam	a899c754f1	Remove whitespace at the end of xmldoc line	2020-06-22 13:03:24 +02:00
Shivam	a94dcc4923	Add xmldoc to MigratableStorage	2020-06-22 12:59:38 +02:00
Shivam	6b14079c0a	InspectCode changes	2020-06-22 12:43:01 +02:00
Shivam	f878388d57	Fix TestMigrationToSeeminglyNestedTarget failing	2020-06-22 11:56:14 +02:00
Shivam	21774b8967	Move static properties to parent class and inherit OsuStorage from it	2020-06-22 11:38:50 +02:00
Shivam	dd9697032c	Introduce new class MigratableStorage	2020-06-16 17:39:20 +02:00
smoogipoo	092f5b6521	Fix incorrect reference + simplify	2020-06-03 17:41:05 +09:00
smoogipoo	8aa8d2c880	Resolve NREs	2020-06-03 16:59:37 +09:00
Dean Herbert	6904d5d247	Remove unnecessary override	2020-05-25 13:12:53 +09:00
Dean Herbert	1977affe7e	Fix OpenInNativeExplorer not working correctly for wrapped storages	2020-05-25 09:27:11 +09:00
Dean Herbert	6ca102bc3f	Attempt delete operations more than once	2020-05-15 13:19:03 +09:00
Dean Herbert	9ed0e8891b	Merge branch 'fix-nested-folder-migration-endless-copy' into migration-ui	2020-05-15 13:04:16 +09:00
Dean Herbert	0690d81bbb	Add protection against migrating to a nested folder	2020-05-15 10:40:26 +09:00
Dean Herbert	25bbb02999	Throw better exceptions from OsuStorage	2020-05-15 10:28:03 +09:00
Dean Herbert	364aa5aa12	Add protection against migrating to a nested folder	2020-05-14 22:45:03 +09:00
Dean Herbert	827d75b152	Revert "Add protection against migrating to a nested folder" ... This reverts commit ef8375b442.	2020-05-14 22:44:27 +09:00
Dean Herbert	ef8375b442	Add protection against migrating to a nested folder	2020-05-14 22:42:42 +09:00
Dean Herbert	06f507496a	Delete migration source if no files exist after completion	2020-05-14 17:41:55 +09:00
Dean Herbert	cb0b25ac55	Throw better exceptions from OsuStorage	2020-05-14 17:41:55 +09:00
Dean Herbert	ad1d050fb4	Throw exception on copy timeout	2020-05-13 20:29:15 +09:00
Dean Herbert	49e616b7e5	Also check for directory presence before migrating	2020-05-13 20:19:14 +09:00
Dean Herbert	9de216f1b4	Merge branch 'master' into migration-backend	2020-05-13 20:15:22 +09:00
Huo Yaoyuan	f07d95ac59	Use IReadOnlyList<T> for TypedListConverter.	2020-05-12 21:49:55 +08:00
Huo Yaoyuan	f7d26688f4	Merge branch 'master' into json-converter	2020-05-12 21:43:02 +08:00
Dean Herbert	e4a23b3e7d	Fix ignored excluding more than top level	2020-05-12 12:39:04 +09:00
Dean Herbert	984f27c107	Add simple retry logic on file copy failure (may be in use)	2020-05-11 21:39:44 +09:00
Dean Herbert	a11be07bb1	Move log storage location after migration complete	2020-05-11 21:39:44 +09:00
Dean Herbert	36640f5dda	Merge branch 'master' into migration-backend	2020-05-11 18:52:36 +09:00
Dean Herbert	384862d48b	Fix incorrect relative paths when using GetStorageForDirectory	2020-05-10 13:20:13 +09:00
Dean Herbert	bbebd26efb	Use DirectoryInfo in more places	2020-05-09 20:13:31 +09:00
Huo Yaoyuan	80a193a616	Use IEnumerable<T> for TypedListConverter.	2020-05-09 17:41:59 +08:00
Huo Yaoyuan	74cbe9306c	Use strongly-typed JsonConerter.	2020-05-09 16:39:46 +08:00
Dean Herbert	7a2020fd45	User copy operation instead of move	2020-05-07 22:12:24 +09:00
Dean Herbert	49a03f1c06	Add basic blocking migration, move not copy	2020-05-07 22:11:57 +09:00
Dean Herbert	7781408643	Update in line with framework storage changes	2020-05-06 18:28:24 +09:00
smoogipoo	d27ca725f9	Use IEnumerable<T> instead	2020-04-22 09:04:44 +09:00
smoogipoo	a702a521f8	Fix not being able to serialise converted beatmaps	2020-04-22 09:04:44 +09:00
smoogipoo	f6f5de7ad1	Allow LineBufferedReader to keep underlying stream open	2020-03-31 10:13:50 +09:00
Dean Herbert	022465f546	Add encoding and import support	2020-03-24 14:51:52 +09:00
Dean Herbert	68ebe98fde	Remove unused GetUnderlyingStream method	2020-03-24 14:08:25 +09:00
Huo Yaoyuan	b14c7bd171	Use type switch in SearializationWriter.	2020-01-27 21:49:34 +08:00
Huo Yaoyuan	40b43b85f1	CA1715: use prefix for generic parameters.	2019-12-17 13:00:25 +08:00
Huo Yaoyuan	e46f6627e4	CA1052: make type static.	2019-12-17 13:00:13 +08:00
Huo Yaoyuan	31cc0d13da	Use 'out var'.	2019-11-12 19:55:26 +08:00
Dean Herbert	35be8f9dfb	Share framework file-exclusion function	2019-10-30 19:34:17 +09:00
Bartłomiej Dach	4b84564f47	Switch casing comparison mode to ordinal ... Switch from InvariantCultureIgnoreCase to OrdinalIgnoreCase when checking file paths in archives for substrings indicating the file can be ignored for performance gains. Co-Authored-By: Dan Balasescu <smoogipoo@smgi.me>	2019-10-11 09:24:41 +02:00
Bartłomiej Dach	57bfa18359	Filter out OS-generated files from archives ... Add a filename ignore list to ZipArchiveReader to filter out superfluous OS-generated files from archives during the import process. In addition to decreasing the size of files imported this allows imports of some incorrectly-constructed archives. An example is the case of having a __MACOSX directory next to a single directory with the actual files - filtering out the former at ZipArchiveReader allows the fallback added in #6170 to work.	2019-10-10 23:54:23 +02:00
Dean Herbert	6268bbcfc8	Merge branch 'master' into beatmap-parsing-fallback-v2	2019-10-03 15:12:21 +08:00
Dean Herbert	d0a4e1e3c2	Force a checksum check before skipping FileStore copy op	2019-09-20 15:00:27 +09:00
Bartłomiej Dach	86588778b1	Implement fallback decoder registration ... After the preparatory introduction of LineBufferedReader, it is now possible to introduce registration of fallback decoders that won't drop input supplied in the first line of the file. A fallback decoder is used when the magic in the first line of the file does not match any of the other known decoders. In such a case, the fallback decoder is constructed and provided a LineBufferedReader instance. The process of matching magic only peeks the first non-empty line, so it is available for re-reading in Decode() using ReadLine(). There can be only one fallback decoder per type; a second attempt of registering a fallback will result in an exception to avoid bugs. To address the issue of parsing failing on badly or non-headered files, set the legacy decoders for Beatmaps and Storyboards as the fallbacks. Due to non-trivial logic, several new, passing unit tests with possible edge cases also included.	2019-09-15 01:28:07 +02:00
Bartłomiej Dach	7b1ff38df7	Implement line-buffered reader ... Add a line-buffered reader decorator operating on StreamReader instances. The decorator has two main operations - PeekLine(), which allows to see the next line in the stream without consuming it, ReadLine(), which consumes and returns the next line in the stream, and ReadToEnd() which reads all the remaining text in the stream (including the unconsumed peeked line). Peeking line-per-line uses an internal queue of lines that have been read ahead from the underlying stream. The addition of the line-buffered reader is a workaround solution to a problem with decoding. At current selecting a decoder works by irreversibly reading the first line from the stream and looking for a magic string that indicates the type of decoder to use. It might however be possible for a file to be valid in format, just missing a header. In such a case a lack of a line-buffered reader makes it impossible to reparse the content of that first line. Introducing it will however allow to peek the first line for magic first. - If magic is found in the first line, GetDecoder() will peek it and use it to return the correct Decoder instance. Note that in the case of JsonBeatmapDecoder the magic is the opening JSON object brace, and therefore must not be consumed. - If magic is not found, the fallback decoder will be able to consume it using ReadLine() in Decode(). This commit additionally contains basic unit tests for the reader. Suggested-by: Aergwyn <aergwyn@t-online.de>	2019-09-15 01:26:15 +02:00
Dean Herbert	5b75060b94	Add test for rollback logic correctly dereferencing files	2019-06-10 16:45:45 +09:00
Dean Herbert	80d65f9a3b	Update resource stores with GetAvailableResources	2019-05-31 14:33:18 +09:00
Dean Herbert	612db31c38	Apply newline additions	2019-04-01 12:16:32 +09:00
Dean Herbert	26d53d06a9	Fix remaining issues	2019-02-28 13:31:40 +09:00
Dean Herbert	a580c17949	Fix codefactor issue	2019-01-24 17:46:09 +09:00
Dean Herbert	8617aaa2a7	Update licence header (and remove year)	2019-01-24 17:43:03 +09:00
smoogipoo	a783fdb501	Add LegacyFileArchiveReader ... Treats files as "archives" for import.	2018-11-28 16:13:16 +09:00
smoogipoo	0982508d26	Rename to LegacyDirectoryArchiveReader	2018-11-28 15:13:27 +09:00
smoogipoo	14ff4bc26a	Move primary key attribute to IHasPrimaryKey	2018-11-28 13:19:23 +09:00
smoogipoo	f27bd3ef3e	OpenTK -> osuTK	2018-11-20 17:14:59 +09:00
smoogipoo	21d5322899	Update with async changes	2018-08-27 17:05:58 +09:00
Dean Herbert	c756a89daf	Standardise path	2018-06-06 20:55:42 +09:00
Dean Herbert	3b03a25ea5	Fix beatmaps with subfolders importing incorrectly on windows ... Closes #2718.	2018-06-05 11:36:44 +09:00
smoogipoo	397d93660a	Don't deep-clone beatmapinfo/control points	2018-05-17 13:17:33 +09:00
Dean Herbert	44a9aa529a	Test CI run with new changes	2018-04-22 03:51:17 +09:00
Dean Herbert	32a74f95a5	Normalize all the line endings	2018-04-13 18:26:38 +09:00
smoogipoo	c29f7a4333	Merge remote-tracking branch 'origin/master' into netstandard	2018-03-24 14:49:46 +09:00
Dean Herbert	b9ef32b09b	Further xmldoc and restoring of Cleanup method	2018-02-15 16:31:42 +09:00
Dean Herbert	1b13be1372	Cleanups and xmldoc additions	2018-02-15 16:24:14 +09:00
Dean Herbert	d3dd31dadb	Make deletion and purging logic even more global	2018-02-15 14:22:39 +09:00
Dean Herbert	d340509b1d	Move ArchiveReaders to a more global namespace ... Also moves delete and action logic to a shared implementation	2018-02-15 14:22:39 +09:00
Dean Herbert	6ff63c2f0c	Move deletion to ArchiveModelImportManager	2018-02-15 14:22:39 +09:00
Dean Herbert	e0d28564d0	Move import logic to shared implementation	2018-02-15 14:22:39 +09:00
Dean Herbert	a738664167	Add interface for database context factory	2018-02-12 23:10:05 +09:00
Dean Herbert	8b37fde15b	Only write when writes occur ... Also add finaliser logic for safety. Also better threading. Also more cleanup.	2018-02-12 19:57:21 +09:00
Dean Herbert	edc3638175	DatabaseWriteUsage	2018-02-12 17:55:11 +09:00
smoogipoo	b500b76407	Merge branch 'master' into update-branch	2018-02-08 23:35:48 +09:00
Dean Herbert	37d393bca0	Update licence headers	2018-01-05 20:21:19 +09:00
smoogipoo	af86220dcc	Merge remote-tracking branch 'origin/master' into update-branch	2018-01-05 12:08:50 +09:00
smoogipoo	5eb2dc3d64	Cleanup some possible CI errors ... Not sure why these aren't being detected on master right now - they should be...	2018-01-05 01:20:24 +09:00
smoogipoo	7a793eaded	Merge master into netstandard	2017-12-27 20:48:53 +09:00
smoogipoo	bdf283a4e1	A bit more cleanup	2017-12-07 15:55:38 +09:00
smoogipoo	245b5f759f	Underscore + lowercase all keys	2017-12-07 14:37:12 +09:00
smoogipoo	d026587a91	Add flag to explicitly serialize the type version	2017-12-07 12:49:16 +09:00