Godot server-client games

[PeterX]

Do I understand Godot correctly? I think:
A game which has been made with Godot, can be easily changed from an offline/local version to a server-client version. (By changing the GUI to HTML5, right?)

Can someone confirm this?

Greetings
Peter

[drummyfish]

Ummm, I don't program in Godot but I'm not sure about that. Note that server-client refers to an architecture in which part of the game runs on the server and part of it runs on potentially many client computers which involves protocols for communication between clients and servers, their synchronization etc. A game running in web browser, i.e. using the HTML5 frontend, doesn't necessarily do this, it simply uses web browser for GUI and I/O as an alternative to for example SDL. The advantage is that such a game may easily be presented on the web and that it's independent of the operating system, but it doesn't mean the game itself works as server-client, it can just be downloaded and played with a web browser.

[PeterX]

I was thinking about writing a network-library. Then I noticed that turning an offline Godot game into an online one is not difficult. I am referring to the GUI/I/O done by HTML5 instead of SDL/Gtk/etc. Exactly as you said.

One could imagine not too complex online games running completely on the server, only doing user in/output on the client computer. You wouldn't have to write completely new network communication code or new GUI code. And you would have a ready-made client.

But there's more than you said: A HTML5 Godot game wouldn't be downloaded completely. I rather would have code that is executed on the server. So it is indeed server-client. Or maybe I'm getting this wrong and a HTML5 Godot game is purely local!?

And yes, there are additional code parts needed for a server, for example handling threads and users.

Greetings
Peter

[dulsi]

HTML5 Godot game is purely local normally. There are ways of doing network play with Godot. I have not explored them so I don't know if they work in HTML5.

[PeterX]

Thanks for the info. Then my assumption was not correct.

I guess that then a network library would make sense.

Greetings
Peter

[drummyfish]

What you're talking about sounds like game streaming which e.g. NVidia tried already -- the game runs on their server and basically just sends rendered images to the client, the client just sends the input. It's a horrible idea in terms of latency, bandwidth, as well as freedom (you own nothing of the game, the game is just a sevice) and bloat. I don't recommend it.

[PeterX]

What you're talking about sounds like game streaming which e.g. NVidia tried already -- the game runs on their server and basically just sends rendered images to the client, the client just sends the input. It's a horrible idea in terms of latency, bandwidth, as well as freedom (you own nothing of the game, the game is just a sevice) and bloat. I don't recommend it.

Yeah, I know what you mean. Like Stadia. I hate that, too. But think of a text-based turn-based strategy game which has a form where you can enter numbers (production, troops, etc.) There the main "functionality"/code is on the server side. Or?

Edit: I have a feeling that this "network library" idea is not such a great idea...

[bzt]

So many interesting topic in one thread!

- Godot HTML5: yep, it's local, meaning generates code that runs in a browser without server parts (webserver is only used to serve the required files). Game logic on client computer.
- Streaming: terrible idea indeed, no wonder Stadia failed and has been canceled. Game logic as well as rendering on the server.
- Networking: the most flexible. Some parts of the game logic on clients, others on server, rendering always happens on client.

But think of a text-based turn-based strategy game which has a form where you can enter numbers (production, troops, etc.) There the main "functionality"/code is on the server side. Or?

Some things to consider using this netowrking-based approach: first, you have to decide how much of your game logic can be pushed to the clients. There are basically two edge cases here:
- Everything happens on the server, sends serialized state to the client and client is basically just for rendering (easy to implement, but not good if your game is fast paced or if you expect many players. Does not scale well, but maybe you don't want it to).
- You do as much as possible on the client, and the server is basically just responsible for synchronizing the game states between clients (difficult to implement properly, you'll need lots of approximations on the client that later gets corrected by the received server data. Advantage is, this kind of server can scale very well as most computation is happening on the clients).
Not finding the proper balance between these will ruin your game. And there's no golden rule here, it all depends what game you're creating.

Keep in mind, HTML5/Javascript+WASM is incapable of doing networking, be careful, running in a browser sucks big time! WASM (or anything else that runs in a browser) can't access any networking interfaces that you'd normally use. You'll need Javascript wrapper libraries for that, which means you are limited to what Javascript can do, and that's it. Either you do small periodic XML requests or you use WebSockets. Raw TCP not possible, and low latency UDP absolutely out of question. Here are some examples:
- fast paced shooter (like Xonotic): you probably want client side game logic with just a small syncing server. Preferably uses UDP, as state changes are small, but constantly updated and needs to be fast. (Forget HTML5, just never gonna work properly)
- turn based strategy: probably server side logic, client only renders as the game "runs" on the server. Preferably with TCP, as state rarely change, but large (many units on the map), so speed isn't an issue, reliability is. (Periodically calling Javascript XML requests could work, but better off with Websockets)

Now Websockets is a real rape on the protocol, huge overhead and waste of resources. Basically it does a HTTP handshake with keep-alive connection and then uses the raw TCP for other, non-HTTP traffic. You'll need a special server to handle those hybrid channels (neither a webserver nor a custom server will do, you'll need something that can do both, hence your server must speak HTTP and SSL/TLS, etc. considerably more complex than a plain TCP server).

If that's not enough, you can't do threading with HTML5/Javascript+WASM. This means you must send / receive all networking data in your main loop, so you cannot process large amounts of data because that would make your game look like it's frozen, which would ruin the game play with irresponsiveness. You might be able to implement some workaround for this, like calling the networking loop using setTimeout(), as that's supported by all browsers and implemented under the hood as a TRUE thread, however in that case the lack of mutexes and other sync primitives is a major PITA, you have to be extremely careful writing your code. My advice is, either you design it in a way that network messages are small and quickly parsable, or design a game where the parsing time doesn't matter, like in a turn based game.

Hope this helps,
bzt

[PeterX]

Thanks for the interesting answer, @bzt!

I see now that my idea is not good. Game code has (in most cases) to be divided between server and client. And there's no simple way to provide a library that does a solution for all kinds of game. Instead the code organisation is different for different games.

Greetings
Peter

[freem]

My message got lost for some reason, so, in short:

Fast paced games do not have to run everything on clients, and should avoid it as it allows clients to easily cheat (wall hacks). This also prevents modifying stuff easily, like in red-eclipse where bots are ran by clients, which prevents AI improvements to land without a new release (and implies that old clients impact on that).
When doing client-server, you might think before hand about how much stuff should be moddable.

Red-eclipse uses a script lang, which is largely insufficient for any serious modding.
Xonotic uses QuakeC, which limits people to an old C variant which never was standard: needs to cope with the limitations of C, plus need to learn to understand the differences versus the standard C.

Unvanquished, which does not sends all data to all clients and do the calculations server-side (while clients have enough info to predict some stuff) uses google-nacl, which replaces quakeC, and is hoped to be replaced by WASM.
From the few researches done on it, WASM does not limits the technologies to write code, it notably does not forces one to use JS or HTML5, but you do need an engine able to understand it, and a toolchain able to generate WASM binaries.

[bzt]

Fast paced games do not have to run everything on clients

Nope, but they should. The reason is simple: sending the user input to the server and waiting for a response introduces latency which kills performance. If more computation happens on the clients, user input is processed in place, then the result is immediate, no network latency involved (but then you'll have to approximate or predict on each client what other players do, and correct it when they receive the latest actual data from the server).

should avoid it as it allows clients to easily cheat (wall hacks)

The part about cheating is true. If game logic happens on the clients, then there's a chance a client might be hacked to cheat. That's one of the reasons why I said pushing game logic to the clients is harder to implement.

This also prevents modifying stuff easily, like in red-eclipse where bots are ran by clients, which prevents AI improvements to land without a new release (and implies that old clients impact on that).

But this one isn't true. It doesn't matter where the bots are handled, it does not make implementing AI more difficult (but other factors make it more difficult to implement more on client side, just not the AI part). Also players must use an up-to-date client no matter what. Both the clients and the server must speak the latest protocol, otherwise there will be incompatibility problems, they won't understand each other.

When doing client-server, you might think before hand about how much stuff should be moddable.

Again, not true. Take for example Minetest, which pushes as much as possible to the clients, yet it is highly moddable.

Red-eclipse uses a script lang, which is largely insufficient for any serious modding.

The scripting language itself doesn't matter, it's the interface that determines how much can be modded in a game. For example, if there's no plugin interface to change the map, then no matter if you use QuakeC, NaCl, WASM whatever, your mods won't be able to destroy the scene (like in Xonotic). And vice versa, if you provide an API to change the map, then it doesn't matter how that's called (what ABI your mods use), you'll be able to change the scene (like in Scorched3D or any voxel game).

Note the difference:
- API: Application Programming Interface, determines what can be done, how much data is sent, what functions are accessible from the mods.
- ABI: Application Binary Interface, determines how an engine call is made and what kind of technology the mods use. This latter has nothing to do with what interfaces are provided for the mods.

Xonotic uses QuakeC, which limits people to an old C variant which never was standard: needs to cope with the limitations of C

Excuse me, what "limitations of C" are you talking about? There are no such thing, people use C for literally everything, from embedded bare-metal, driver and kernel programming to high level user applications like SDL games. It is the most portable language ever created.

It is a proven fact, that as long as a language supports sequence, conditionals and loops (and C does), then all kind of algorithms can be implemented in that language (if I'm not mistaken, then it was Nicolaus Wirth who proved this). Some languages are easier to use than others, but it's never the language that might limit you.

Unvanquished, which does not sends all data to all clients and do the calculations server-side (while clients have enough info to predict some stuff) uses google-nacl, which replaces quakeC, and is hoped to be replaced by WASM.

Again, these are just ABI implementation details, they won't change the API. If you hope that using a different language or bytecode interpreter would solve your issue then you're on the wrong path. It might give you better performance on the clients, but that's all.

From the few researches done on it, WASM does not limits the technologies to write code

Neither does other languages, like QuakeC. Again, it does not matter what kind of ABI you use, it's the API that limits your modding capability.

it notably does not forces one to use JS or HTML5

On the contrary. WASM forces you to use JS when run in a browser (using the emscripten ABI). It is true that WASM could be used outside of the browser (using a different ABI and API called WASI), but that's fundamentally incompatible with everything you'd normally associate with WebAssembly.

Simply put: if you compile your WASM code for WASI, then you won't be able to run it in a browser. And vice versa, if you compile it for emscripten, then you'll never be able to run it without Javascript and outside of a browser.

There was some work done trying to replace emscripten ABI with WASI ABI (or at least make them compatible), but the effort miserably failed after years of trying, and it's now labelled "wontfix". Those ABIs are just way too incompatible by design, they can never work together.

but you do need an engine able to understand it, and a toolchain able to generate WASM binaries.

Yep, but as I've said it's a little bit more complicated than that, because the WASM has multiple competing ABI standards.

Here's a good, easily embeddable WASM interpreter written in C. Here you can clearly see, that the wax interpreter is for running bytecode compiled for the WASI interface, and wace is for the ones compiled for emscripten (note this just implements the ABI, it does not provide the environment normally found in browsers!!!). The two are incompatible. The first needs lots of hooks in the engine (wax provides some of these implemented atop of POSIX libc), and the latter needs Javascript wrapper libraries running in a browser to provide the WASM interfaces (wace does not provide these at all).

Cheers,
bzt

[freem]

Nope, but they should

I agree that unloading the server can help a lot with global performances, but as you pointed out, the difficulty to do that is not exactly low. This, the difficulty and maintenance problems, are really what I wanted to focus on, as well as giving some examples that I know because I played (and sometimes even contributed to) them.
I do not know minetest at all, so I can not talk about it.

But this one isn't true. It doesn't matter where the bots are handled, it does not make implementing AI more difficult

It does make them more difficult to implement if you do not want to make it easy for players to cheat, in this case, by implementing AIs that deliberately suck one way or another if they're not in your team (which is fixable, by putting all AIs in their teammate's team, but this is just one example) or implementing an AI that cheats (even more, as this is what most games seem to do to implement better AIs... and this sucks big time) if in your own team.

Again, not true. Take for example Minetest, which pushes as much as possible to the clients, yet it is highly moddable.

One of the things one have to check when game is moddable and runs logic on clients is notably to check that the clients are all in sync (compatible API).
While this is not technically hard to do when you trust the clients (and to trust FOSS clients, you can not rely on hashes, as they can send whatever they want after small tinkering), this have implications on the long-term, on maintenance. Except, of course, if you had you API correct and complete on 1st version.

You say "as much as possible", do you know which parts are not pushed on clients?

Excuse me, what "limitations of C" are you talking about? There are no such thing, people use C for literally everything, from embedded bare-metal
...
Some languages are easier to use than others, but it's never the language that might limit you.

That could have been done in assembly, too, except for the portability thing. The limitations of C are not from performance or possibilty perspectives, but from maintenance ones.
About proven facts, there's one about the fact C is not very helpful to avoid resource-related bugs (resource might be memory, connections to remote machine, etc). Yes, skilled C programmers will avoid many of the pitfalls, but they won't avoid all of them. What other languages can provide is not avoiding those problems neither, but to make it easier to get their number lower.
As you say: some languages are easier to use than others. C is definitely not the simpler lang to write correct code (but it can do that, I'm running a kernel that's written in C, and problems are pretty rare. They do happen, though, like some freezes on some hardware in some conditions that I do not know... might be bugs in drivers, or failing hardware, for what I know).
I insist on the correct word. And before you say it, neither is C++.

Neither does other languages, like QuakeC. Again, it does not matter what kind of ABI you use, it's the API that limits your modding capability.

The language itself does not limits people, technically, as you can always write an interpreter or a compiler (as long as you have the specs) but this is in practice a limitation, because doing that would require tremendous amount of time that would not be spent on the real project.
This remark applies to embedding any interpreter though (for example, unvanquished's usage of NaCl does have limitations. Despite the thing being open source, it only ever supported x86, x86_64 and arm architectures (not sure which arm archs, though), and unv's team was never able to make it run on arm anyway).

If you hope that using a different language or bytecode interpreter would solve your issue then you're on the wrong path. It might give you better performance on the clients, but that's all.

This depends on the issue we're talking about. You name yourself the performance issue, but there's also the maintenance issue, the portability issue, or the fact that restricting oneself to i.e. quakeC (xonotic's case) can be a problem from a human PoV (because one have to learn the subtleties of this C variant, in this case).
For example, one could say using Win32 API is not going to limit you, as you can always reimplement it. Some people are working on this since more than a few years, though, and they're still not close to have finished the job.

Simply put: if you compile your WASM code for WASI, then you won't be able to run it in a browser.

This is what I was talking about. Yes, ofc it's a different, less common use, of the technology, but it's still a valid and existing one. And running everything in a browser is a pretty ugly idea, as it means your game won't run with less than 2 gigabytes of RAM, roughly, and this, in 2022. In 2025, it might become 5Gigs, or more.
Not to mention the constantly moving target (web standards are far from being static, this means the moment you stop updating your game, few years after there will be almost nothing to salvage of it).

Yep, but as I've said it's a little bit more complicated than that, because the WASM has multiple competing ABI standards.
Here's a good, easily embeddable WASM interpreter written in C.

This is new to me. Thanks for the info (and I'll take a look at this interpreter, out of curiosity).

[bzt]

It does make them more difficult to implement if you do not want to make it easy for players to cheat

As I've said, implementing the AI isn't more difficult. But implementing other things (like anti-cheating) is.

One of the things one have to check when game is moddable and runs logic on clients is notably to check that the clients are all in sync (compatible API).

Nope, one could always use "the server is always authentic" paradigm, and eliminate the need of sync entirely. It might result in glitches if the client's approximation and the true game state diverges, that's why it's not used often, but possible. (Although pretty common in fast paced games like xonotic because there state updates are sent and received frequently, so even if the approximation diverges, that bad data is not used for long, as a new update packet arrives pretty soon.)

You say "as much as possible", do you know which parts are not pushed on clients?

Depends on the game, as I've said. I've given two edge case examples, one where all game logic is on the server, the client is just basically for rendering, and another one where all the game logic is on the clients, and server is just used for state sync. This depends entirely on what kind of game you're designing.

The limitations of C are not from performance or possibilty perspectives, but from maintenance ones.

Considering that I can easily compile 30 and 40 years old ANSI C code with the latest compiler without any probs, I would argue otherwise. I cannot say that about Python, Rust, Go, Java, Javascript, etc. or about any so called "modern" language. (Yes, there could be issues with some non-standard compiler specific C extensions, but those are not related to the core language.)

About proven facts, there's one about the fact C is not very helpful to avoid resource-related bugs (resource might be memory, connections to remote machine, etc).

Neither are other languages. Any average C++ code is full of memory errors, but I could mention Cyberpunk 2077 too as an outstanding example for bugs. And Rust's approach introduces more problems than solutions. I haven't seen any useful data structure (like graph, double linked list, etc.) implementation in Rust which isn't using "unsafe" all the time, essentially turning off all the "resource protection" because it's just not working.

Yes, skilled C programmers will avoid many of the pitfalls, but they won't avoid all of them.

There's a need for testing and debugging, no matter the language used.

What other languages can provide is not avoiding those problems neither, but to make it easier to get their number lower.

I'd argue that's not true. They just introduce different kind of problems.

The language itself does not limits people, technically, as you can always write an interpreter or a compiler (as long as you have the specs) but this is in practice a limitation, because doing that would require tremendous amount of time that would not be spent on the real project.

There are plenty of ready-to-use compilers and interpreters, pick one. Lua is very popular because it's easy to integrate.

If you hope that using a different language or bytecode interpreter would solve your issue then you're on the wrong path. It might give you better performance on the clients, but that's all.

This depends on the issue we're talking about. You name yourself the performance issue, but there's also the maintenance issue, the portability issue, or the fact that restricting oneself to i.e. quakeC (xonotic's case) can be a problem from a human PoV (because one have to learn the subtleties of this C variant, in this case).
For example, one could say using Win32 API is not going to limit you

Again, you're mixing languages with the API. And yes, Win32 API limits you big time, have you ever tried to implement POSIX fork atop of it? Almost impossible...

And running everything in a browser is a pretty ugly idea, as it means your game won't run with less than 2 gigabytes of RAM, roughly, and this, in 2022. In 2025, it might become 5Gigs, or more.

On that we agree. But it's not 2G, rather 512M per tab, and what's more irritating to me is that you must provide Javascript wrappers for all WASM functionality, which kills performance big time, and unavoidably inherits all the limitations of Javascript. You just can't do lots of things, like using an UDP socket or listing files in a directory on the computer for example.

Cheers,
bzt