This fixes the recreation of all panels when a download completes.
Also fixes NullReference when you download without ever opening the details of one Set.
This is important when using dynamic compiling to rapidly iterate. Until we actually split projects out into pieces (like the abstract ruleset project we have talked about) there is no advantage to using internal in the osu! game code.