0 Testing web server full stack with osu!

Salman Alshamrani edited this page 2024-10-26 23:00:21 -04:00

Table of Contents

• Prerequisites
• Initial setup
• Basics
• Score submission
• Replays
• Running

This is a guide to help ease the setup of a full development environment with

• osu!
• osu-web
• osu-server-spectator

This will allow testing all operations including multiplayer, spectating, account creation etc.

Prerequisites

• osu! in a compiling state (https://github.com/ppy/osu#developing-osu)
• osu-web setup using docker (https://github.com/ppy/osu-web/blob/master/SETUP.md#2-using-docker)
  • using an old osu-web clone may result in running under dirty/outdated files (such as a dirty .env), it is recommended to use a fresh clone if you're not experienced with web setups.
• osu-server-spectator in a compiling state (https://github.com/ppy/osu-server-spectator)
• All three projects should be in the same parent directory (as osu / osu-web / osu-server-spectator)

Initial setup

Basics

Make sure you have setup a client key. You can use it to populate DevelopmentEndpointConfiguration.cs in your osu! checkout (along with the endpoint URLs):

diff --git a/osu.Game/Online/DevelopmentEndpointConfiguration.cs b/osu.Game/Online/DevelopmentEndpointConfiguration.cs
index 5f3c353f4d..284aba6e98 100644
--- a/osu.Game/Online/DevelopmentEndpointConfiguration.cs
+++ b/osu.Game/Online/DevelopmentEndpointConfiguration.cs
@@ -7,12 +7,14 @@ public class DevelopmentEndpointConfiguration : EndpointConfiguration
     {
         public DevelopmentEndpointConfiguration()
         {
-            WebsiteRootUrl = APIEndpointUrl = @"https://dev.ppy.sh";
-            APIClientSecret = @"3LP2mhUrV89xxzD1YKNndXHEhWWCRLPNKioZ9ymT";
-            APIClientID = "5";
-            SpectatorEndpointUrl = $@"{APIEndpointUrl}/signalr/spectator";
-            MultiplayerEndpointUrl = $@"{APIEndpointUrl}/signalr/multiplayer";
-            MetadataEndpointUrl = $@"{APIEndpointUrl}/signalr/metadata";
+            WebsiteRootUrl = APIEndpointUrl = @"http://localhost:8080";
+            const string spectator_server_root_url = @"http://localhost:80";
+
+            APIClientSecret = @"<generated client secret>";
+            APIClientID = "<generated client ID number>";
+            SpectatorEndpointUrl = $"{spectator_server_root_url}/spectator";
+            MultiplayerEndpointUrl = $"{spectator_server_root_url}/multiplayer";
+            MetadataEndpointUrl = $"{spectator_server_root_url}/metadata";
         }
     }
 }

In osu-server-spectator, update the client ID in ConfigureJwtBearerOptions:

diff --git a/osu.Server.Spectator/Authentication/ConfigureJwtBearerOptions.cs b/osu.Server.Spectator/Authentication/ConfigureJwtBearerOptions.cs
index 6dcdc5c..f3f6b3f 100644
--- a/osu.Server.Spectator/Authentication/ConfigureJwtBearerOptions.cs
+++ b/osu.Server.Spectator/Authentication/ConfigureJwtBearerOptions.cs
@@ -30,7 +30,7 @@ namespace osu.Server.Spectator.Authentication
             options.TokenValidationParameters = new TokenValidationParameters
             {
                 IssuerSigningKey = new RsaSecurityKey(rsa),
-                ValidAudience = "5", // should match the client ID assigned to osu! in the osu-web target deploy.
+                ValidAudience = "<generated client ID number>", // should match the client ID assigned to osu! in the osu-web target deploy.
                 // TODO: figure out why this isn't included in the token.
                 ValidateIssuer = false,
                 ValidIssuer = "https://osu.ppy.sh/"

Copy the set up osu!web oauth public key across to the spectator project:

$ cp osu-web/storage/oauth-public.key osu-server-spectator/osu.Server.Spectator/oauth-public.key

Now you can run osu-server-spectator in release configuration (debug/development should be avoided as it assigns random user IDs to any connected client, rather than their actual user IDs).

By default, osu-server-spectator will bind to local port 80. This may fail on desktop PCs (with a System.Net.Sockets.SocketException: Permission denied error), in which case you should change the port to a different one:

diff --git a/osu.Server.Spectator/Program.cs b/osu.Server.Spectator/Program.cs
index d21cdd2..50bcb80 100644
--- a/osu.Server.Spectator/Program.cs
+++ b/osu.Server.Spectator/Program.cs
@@ -55,8 +55,8 @@ namespace osu.Server.Spectator
                            webBuilder.UseStartup<Startup>();
 #endif
 
-                           webBuilder.UseUrls(urls: new[] { "http://*:80" });
+                           webBuilder.UseUrls(urls: new[] { "http://*:8081" });
                        });
         }
     }

If the above change is made, spectatorServerRootUrl in DevelopmentEndpointConfiguration in the client must also be updated to match.

To make osu! work without https, you will need to adjust a few files for the time being:

diff --git a/osu.Game/Online/API/OsuJsonWebRequest.cs b/osu.Game/Online/API/OsuJsonWebRequest.cs
index 4a45a8b261..9bb8dd9628 100644
--- a/osu.Game/Online/API/OsuJsonWebRequest.cs
+++ b/osu.Game/Online/API/OsuJsonWebRequest.cs
@@ -10,10 +10,12 @@ public class OsuJsonWebRequest<T> : JsonWebRequest<T>
         public OsuJsonWebRequest(string uri)
             : base(uri)
         {
+            AllowInsecureRequests = true;
         }

         public OsuJsonWebRequest()
         {
+            AllowInsecureRequests = true;
         }

         protected override string UserAgent => "osu!";
diff --git a/osu.Game/Online/API/OsuWebRequest.cs b/osu.Game/Online/API/OsuWebRequest.cs
index 1d27899473..f062c534ae 100644
--- a/osu.Game/Online/API/OsuWebRequest.cs
+++ b/osu.Game/Online/API/OsuWebRequest.cs
@@ -10,6 +10,7 @@ public class OsuWebRequest : WebRequest
         public OsuWebRequest(string uri)
             : base(uri)
         {
+            AllowInsecureRequests = true;
         }

         public OsuWebRequest()
diff --git a/osu.Game/Online/API/RegistrationRequest.cs b/osu.Game/Online/API/RegistrationRequest.cs
index f650e5c93b..56949dfeac 100644
--- a/osu.Game/Online/API/RegistrationRequest.cs
+++ b/osu.Game/Online/API/RegistrationRequest.cs
@@ -11,6 +11,11 @@ public class RegistrationRequest : OsuWebRequest
         internal string Email = string.Empty;
         internal string Password = string.Empty;

+        public RegistrationRequest()
+        {
+            AllowInsecureRequests = true;
+        }
+
         protected override void PrePerform()
         {
             AddParameter("user[username]", Username);

At this point, osu will connect to and consume osu-web API endpoints, as well connect to the osu-server-spectator instance. However, more specialised flows such as score submissions will not work yet.

Score submission

In order to be able to submit any scores in such a local environment, set the following environment variable in osu-web/.env:

CLIENT_CHECK_VERSION=false

This skips the client compatibility check on the osu-web side that would normally happen in production to ensure that the build the score was set on is a known build.

Also, in order for the submitted score to be processed and applied to user statistics, you will need to have an instance of osu-queue-score-statistics running in the stack (in queue watch mode).

Replays

osu-server-spectator is responsible for storing replays for all scores captured using the client. To enable the replay-capturing flow in a way that works with osu-web and the client, the following steps must be performed:

In osu-web, enable local replay storage by adding

SCORE_REPLAYS_STORAGE=local

to osu-web/.env.

In osu-server-spectator, apply the following diff:

diff --git a/osu.Server.Spectator/Extensions/ServiceCollectionExtensions.cs b/osu.Server.Spectator/Extensions/ServiceCollectionExtensions.cs index 1100f64..403f1f6 100644
--- a/osu.Server.Spectator/Extensions/ServiceCollectionExtensions.cs
+++ b/osu.Server.Spectator/Extensions/ServiceCollectionExtensions.cs
@@ -22,7 +22,7 @@ namespace osu.Server.Spectator.Extensions
                                     .AddSingleton<EntityStore<ServerMultiplayerRoom>>()
                                     .AddSingleton<GracefulShutdownManager>()
                                     .AddSingleton<MetadataBroadcaster>()
-                                    .AddSingleton<IScoreStorage, S3ScoreStorage>()
+                                    .AddSingleton<IScoreStorage, FileScoreStorage>()
                                     .AddSingleton<ScoreUploader>()
                                     .AddSingleton<IScoreProcessedSubscriber, ScoreProcessedSubscriber>();
         }
diff --git a/osu.Server.Spectator/Properties/launchSettings.json b/osu.Server.Spectator/Properties/launchSettings.json
index 133a303..bdf3f20 100644
--- a/osu.Server.Spectator/Properties/launchSettings.json
+++ b/osu.Server.Spectator/Properties/launchSettings.json
@@ -15,7 +15,9 @@
       "launchUrl": "spectator",
       "applicationUrl": "http://localhost:5009",
       "environmentVariables": {
-        "ASPNETCORE_ENVIRONMENT": "Development"
+        "ASPNETCORE_ENVIRONMENT": "Development",
+        "SAVE_REPLAYS": "1",
+        "REPLAYS_PATH": "${OSU_WEB_DIRECTORY}/public/uploads-solo-replay"
       }
     }
   }

where ${OSU_WEB_DIRECTORY} points to the osu-web directory root.

In osu, apply the following change:

diff --git a/osu.Game/Online/API/APIDownloadRequest.cs b/osu.Game/Online/API/APIDownloadRequest.cs
index c48372278a..ffa4a4c025 100644
--- a/osu.Game/Online/API/APIDownloadRequest.cs
+++ b/osu.Game/Online/API/APIDownloadRequest.cs
@@ -31,6 +31,7 @@ protected override WebRequest CreateWebRequest()
 
             var request = new FileWebRequest(filename, Uri);
             request.DownloadProgress += request_Progress;
+            request.AllowInsecureRequests = true;
             return request;
         }
 

to allow downloading the replay over HTTP.

After this, it should be possible to download replays set via the lazer client and recorded by osu-server-spectator, both via osu-web and the lazer client.

Running

• start osu-web

$ cd osu-web
$ docker-composer up

• start osu-server-spectator in release configuration (release config is required to connect to the osu-web instance)

$ cd osu-server-spectator
$ dotnet run -c Release

• start osu! in debug configuration (via IDE or otherwise)
  • you can use the --debug-client-id={number} argument with different IDs for running as many osu! instances for testing as desired.