OD - Fixing the Multiplayer

[Bertram]

Hi again,

@hwoarangmy:

Concerning multiplayer, I have made a quick GUI based on the level chooser that works. I guess later, when a new player will install the game, he should be prompted some stuff like language, nickname... For now, I have added the nickname edit and the client IP edit in the multiplayer screen.
It seems to load more or less. I've fixed some issues with seat handling but it is not playable yet. I've tried the Test level but adding tiles takes more than 30 minutes (seen with logs). I will have a look to see if something can be done about it. But even when everything is loaded, the game does not work. I will try to make a smaller map (lets say 50x50) and try to figure out what is going on.

Did you find a way to readd new player listening from the server without forcing it to be dedicated and without using SFML net? I'm eager to have a look at that anyway.

@MCMic:
I saw the trap cannon fire boulders several times on latest git without crashing. Is it still doing it for you?

Best regards,

[hwoarangmy]

Did you find a way to readd new player listening from the server without forcing it to be dedicated and without using SFML net? I'm eager to have a look at that anyway.

In fact, there were 2 issues with that. The first one is that the accept function waits forever for a new player to connect. As an easy fix, I just check is there is an empty player seat before calling accept. If not, I just skip the call
The other issue was the receive function listening for chat messages. There, I used ioctrlwinsock function that allows to check if data is available before calling recv (I've not tried but it seems there is also ioctrl function that does the same for windows.

But there are more problems to deal with before multiplayer works. I've fixed some issues with Seat attribution. I've seen that the server sends all the map data to the clients (including tiles and all). In fact, client do not need to load the map at all. For the test map, It took more than 30 minutes on my decent computer to do that. I created a 50x50 map which is much faster to load. But this loading time is clearly not acceptable so we should decide if we prefer to make sure that client and server have to select the same map to play together and accept buggy behaviour if the 2 maps are different.

So far, I could not make multiplayer work. I didn't have time to investigate but after the game is launched, the client do not work (even if the server seems ok). The client have no light and it is hard to navigate. Moreover, because I have skipped the map loading, the client do not start over his dungeon heart so it is hard to find it.

But the changes are heavier than I thought so I think I will try to use SFML network now that I am at it. We should also decide if we keep this behaviour where there needs to be has many players as player seats or if we allow to use player seats with AI player (but this would recquire a new menu)

[Bertram]

Hi hwoarangmy,

In fact, there were 2 issues with that. The first one is that the accept function waits forever for a new player to connect. As an easy fix, I just check is there is an empty player seat before calling accept. If not, I just skip the call
The other issue was the receive function listening for chat messages. There, I used ioctrlwinsock function that allows to check if data is available before calling recv (I've not tried but it seems there is also ioctrl function that does the same for windows.
[...]
So far, I could not make multiplayer work. I didn't have time to investigate but after the game is launched, the client do not work (even if the server seems ok). The client have no light and it is hard to navigate. Moreover, because I have skipped the map loading, the client do not start over his dungeon heart so it is hard to find it.
But the changes are heavier than I thought so I think I will try to use SFML network now that I am at it. We should also decide if we keep this behaviour where there needs to be has many players as player seats or if we allow to use player seats with AI player (but this would recquire a new menu)

Cool to hear. Do you want to start by merge that or do you prefer have a go with SFML net first?

But there are more problems to deal with before multiplayer works. I've fixed some issues with Seat attribution. I've seen that the server sends all the map data to the clients (including tiles and all). In fact, client do not need to load the map at all. For the test map, It took more than 30 minutes on my decent computer to do that. I created a 50x50 map which is much faster to load. But this loading time is clearly not acceptable so we should decide if we prefer to make sure that client and server have to select the same map to play together and accept buggy behaviour if the 2 maps are different.

I see. Sending all the tiles to all clients is unacceptable and irrelevant anyway. So yes, let's make the client and the server try to load the same filename and cope with this. Later on, we always can send a md5sum to the clients so they can check it is actually the same file.

Best of luck for the digging and have some nice holidays.

Regards,

[hwoarangmy]

Cool to hear. Do you want to start by merge that or do you prefer have a go with SFML net first?

I will upload as soon as I have something working. I am currently trying to stick as close as possible to the actual code. But since I have my nose there, I will try to use SMFL (from what I've seen, it's not very hard).

But after that, I will try to create a dedicated thread for the server to come closer to the full server paradigm. Get ready, I'm going to break everything

Best of luck for the digging and have some nice holidays.

Thanks

[Bertram]

I will upload as soon as I have something working. I am currently trying to stick as close as possible to the actual code. But since I have my nose there, I will try to use SMFL (from what I've seen, it's not very hard).
But after that, I will try to create a dedicated thread for the server to come closer to the full server paradigm. Get ready, I'm going to break everything

I've not yet looked at that at all, but hopefully SFML will help you "hiding" the server thread behind a sane API.

Regards,

[hwoarangmy]

I've not yet looked at that at all, but hopefully SFML will help you "hiding" the server thread behind a sane API.

From what I've seen in SFML code, selectors (what you are talking about to handle many clients in the same thread) just call C standard function "select". So we could handle network even without including SFML network. And from what I've seen, there is no thread created.

But I will include it because :
1 - We already have SFML (no need to add another library)
2 - It is a well tested API that have classes that help handling network stuff (like endianness)
3 - It is a high level API that will handle OS/hardware specific stuff (so we don't have to care about sockets, winsocks and their friends)

Concerning the server thread, when I will add it (to update the gamemap), shall I use boost ? I have seen no thread creation in OD code so I'm not sure what is the proper way to do so...

[Bertram]

Hi there,

From what I've seen in SFML code, selectors (what you are talking about to handle many clients in the same thread) just call C standard function "select". So we could handle network even without including SFML network. And from what I've seen, there is no thread created.

But I will include it because :
1 - We already have SFML (no need to add another library)
2 - It is a well tested API that have classes that help handling network stuff (like endianness)
3 - It is a high level API that will handle OS/hardware specific stuff (so we don't have to care about sockets, winsocks and their friends)

Well, this is more or less what I had in mind so you have my full support about that.

Concerning the server thread, when I will add it (to update the gamemap), shall I use boost ? I have seen no thread creation in OD code so I'm not sure what is the proper way to do so...

While we're speaking about this, I must say I spent quite some time removing the old threaded code and you may see that not too far away in the git history. Now, about the server/client threading, I'd be in favour of using SFML in this area also. What do think of this example, for instance?
https://github.com/LaurentGomila/SFML/w ... at-Example

Regards,

[oln]

Any reason not to use the STL for threads?

[Bertram]

In that specific case, maybe yes, as we'd use thread only for the server main loop which would use SFML code also.
In the end, it doesn't matter much as long as it is readable.

[hwoarangmy]

What do think of this example, for instance?

Good to see how threads are supposed to be used. The example for what I'm going to do :
http://www.sfml-dev.org/tutorials/2.1/n ... socket.php

Any reason not to use the STL for threads?

No, it is just that I have seen no thread creation at all in the code so I was not sure what was the proper way in this project. But I guess that SFML, Boost or STL are all well tested APIs so it doesn't matter much which one is used... And as I am using SFML for the server network, I will also create use it for the server thread creation.

And, to answer Paul (who as visibly removed his message), yes I am on holydays but as I have internet, I use it to discuss the proper implementation I shall use when I'll be back

[hwoarangmy]

I take the opportunity of this post to summarize what to do about multiplayer fix. In most of the cases, I will say what I'm planning to do for the next patch and what should be done in a next release.
Most of the things that I will say are in my opinion and are there to be discussed so if you think something is wrong/can be improved, please don't hesitate to say

Moreover, If you think that something that I have planned in the next release is important and should be done sooner, it is the good place to say it

1 - How should the player enter his nicknames ?
Currently : I added an Editbox in a new multiplayer menu
Later : When the player first launches the game, it could be nice to have a screen asking for his configuration : language, nickname, ...

2 - In the current code, for multiplayer, the players are identified within the code by their nicknames. That means that when Paul drops a creature, a message is sent to every connected player saying that Paul dropped a creature. The problem is that it supposes that every nickname is unique, which is very unlikely. It will probably cause crashes or at least strange behaviours if it happens.
Identify each player through a unique number given by the server when they connect (would allow to switch player's color before launching game)

3 - How to handle 2 identical nicknames ?
Currently : Ignore
Later : ?

4 - Should the server send the whole map to every client as it does now ?
Currently : No, it should just check that the map name is the same as the one he opened
Later : No, it should just check that the map name is the same as the one he opened and maybe use a md5 ?

5 - How to handle empty player seats ?
Currently : every player seat as to be filled to start the game.
Later : Add a GUI screen to allow adding AI instead of player ?

6 - Shall we use SFML network now ?
Yes. As I said, the changes for making multiplayer work are heavier as I thought and will require many debug time. So I guess it would be a good thing to use SFML now instead of debugging standard sockets and have untested side effects when we integrate SFML network.

7 - Shall I make dedicated classes for Server and Client network ?
Yes. In fact, it's almost already done so I will upload it when I'm back

Concerning the git commits, I think I will commit the SFML network integration and server/client dedicated class first firstly so that it is easier to see which changes concerns what.
Then, I will try to make multiplayer work regarding what we have decided about the previous points

[paul424]

Looks like Bertram have gone for good on holidays. at we only have each other ... ehem in terms of coding of course :P ,
I could have asked the same , if you be nice to tell something about RM split ( the other forum thread ) ...

6 - Shall we use SFML network now ?
Yes. As I said, the changes for making multiplayer work are heavier as I thought and will require many debug time. So I guess it would be a good thing to use SFML now instead of debugging standard sockets and have untested side effects when we integrate SFML network

I always don;'t mind using external libraries , if it simplifes or speeds up the coding ...

[hwoarangmy]

I have uploaded on my github account some changes (as always, I have re-forked Bertram's repo before uploading).
I have created classes for Client and server. They inherit classes that handle communication using SFML. The communication classes have responsability to handle data as packets (they only notify to the server complete packets - exactly 1 per packet sent) and they can be used with operators << and >>.
Most of the communication classes have not been tested but the single player game is still working. I did not test the client/server because I had no simple way to do it. Now, I'm going to try to fix multiplayer and I will check everything works with these new classes. I think this changes should be merged to avoid a too big merging when multiplayer will be fixed

[Bertram]

Nice work done so far

There is indeed no simple way to test all that atm since things are still quicksandy in that area, but I'm sure we have now a good basis for basic multiplayer gamepay.

Regards,

[hwoarangmy]

There is indeed no simple way to test all that atm since things are still quicksandy in that area, but I'm sure we have now a good basis for basic multiplayer gamepay.

I've done a menu to setup multiplayer game acording to the points I was talking about in this thread. Now, I can launch a game and it works \o/
For now, I still have some problems implementing the things that were not done (adding creatures, rooms, updating gold for clients...). So far (local network), it doesn't look too bad. The server and the client are almost synchronized, the creatures do the same things.
I have made a smaller map that I will upload when multiplayer works.
Now, I am having a bad time with the goals. Problem is that only the server should compute if a goal is met or not (because if there is fog of war, the clients will not know how many creatures remain or whatever). But as the goals are done (they include pointers to subgoals), it not easy to inform the clients about which goals are met or not

[Bertram]

I've done a menu to setup multiplayer game acording to the points I was talking about in this thread. Now, I can launch a game and it works \o/
For now, I still have some problems implementing the things that were not done (adding creatures, rooms, updating gold for clients...). So far (local network), it doesn't look too bad. The server and the client are almost synchronized, the creatures do the same things.

You made my day. \^o^/

I have made a smaller map that I will upload when multiplayer works.

Do you mean the server is sending all the tiles atm?

Now, I am having a bad time with the goals. Problem is that only the server should compute if a goal is met or not (because if there is fog of war, the clients will not know how many creatures remain or whatever). But as the goals are done (they include pointers to subgoals), it not easy to inform the clients about which goals are met or not

Ah I see. For all I could so far, the goal should be sent added/modified and removed only by the server. To make it even more simple, I'd even propose that the client doesn't handle sub-goals at all, letting the server do all the logic and replace a goal by another according to the case.
This means we'll need to move the goal logic to be server-based only. For now, you can let this aside if it's hurting, let's make the core game work first.
In any case, I'd propose we define a small packet list based on the goal events sent by the server to the clients first, so one have a clear set of things to implement. What do you think?

Regards,

[hwoarangmy]

Do you mean the server is sending all the tiles atm?

Not anymore When the server is started, I added as an argument the name of the map (at each map, the server has to be stopped and then started) so that he can laod it. That way, when a client connects, he has to send the name of the map he loaded and if it doesn't match, the server disconnects him and waits for another client.
But with the big map, in debug mode with 2 OD games launched, it lags too much
Now that we talk about it, I have seen in gui.cpp in function bool Gui::serverButtonPressed(const CEGUI::EventArgs& e), the server is started. As far as I've seen, it comes from an ingame menu but I couldn't see how it works. Is it some dead code or is it used ?

[quote="Bertram"]Ah I see. For all I could so far, the goal should be sent added/modified and removed only by the server. To make it even more simple, I'd even propose that the client doesn't handle sub-goals at all, letting the server do all the logic and replace a goal by another according to the case./quote]Good idea. I will put the goals somewhere in the server side so that the server only has to send the minimum needed to display the goals

[Bertram]

But with the big map, in debug mode with 2 OD games launched, it lags too much

Ah, yes, indeed.

Now that we talk about it, I have seen in gui.cpp in function bool Gui::serverButtonPressed(const CEGUI::EventArgs& e), the server is started. As far as I've seen, it comes from an ingame menu but I couldn't see how it works. Is it some dead code or is it used ?

Ancient gui button, I guess. We should later remove this once a multiplayer mode is officially added. (I guess we're not far from it thanks to your work)

Good idea. I will put the goals somewhere in the server side so that the server only has to send the minimum needed to display the goals

Thanks for taking care of that as well.

Btw, I synced the sourceforge repo with your latest changes, and I am half-way done with updating the in-game window skin. The graphics are done, but the look'n'feel side is always a bit made of trial and errors.

Regards,

[hwoarangmy]

Hi,

I've completed first step on fixing multiplayer and sent a new pull request . BTW, I've added a new small map to test it (it also works with the big one too).
I've renamed the test map to TestSingleplayer and TestMultiplayer.

To try it, I've added a multiplayer menu

I've tested for some time and it seems to work well. If someone could test it with windows, it could be cool. When a multiplayer map is launched in single player, every other player than the first is set to AI player.
There are still some minor bugs I haven't fixed yet :
- Workers sometimes work on wrong direction on client (they are looking east while destroying north tile for example)
- Gold tiles mined are golden in client and not in server
- Only full claimed tiles are notified
- Dead creatures do not disapear on client
- Traps appear for every player (I guess we can change that when we would have decided what we want)
- I have not tested chat (I have seen the feature and I think it is supposed to work through the console but I think we should decide how it will be used in game)

Now, I will have a look to make dead creatures disappear. Then, I will try to make the local player work as a normal client so that their behaviour is as close as possible

As the changes are quite heavy, I will try to summarize what I've done :
- The client and the server both load the level. They check that the map name is the same.
- The server only notifies changes after the map load (loading what is in the level is up to the client).
- There is no more command synchronisation when the server and the client communicates. I guess it was there because the server used to send the whole map to the clients so synchronisation was probably needed. But as our commands are much lower, I removed it. We can always use it again for specific cases if it is needed (but it will slower the game much).
- There is a synchronisation between server and clients at every turn. Until every client has sent an ack to the server with the current turn, the next one will be delayed.
- I've tried to set up the full server paradigm. That means that the clients do not notify anymore what they do. Now, they ask what they want, the server computes what happens and notifies the client what he does. For example, when the client clicks on a creature, a message is sent saying that the client wants to pick it. The server checks if the action is allowed and, if he decides it is ok, he sends a messages to the clients saying that the creature has been picked up

[Bertram]

Hi Hwoarangmy,

This is quite some excellent work, and done so quickly! I'm impressed.

@Everyone:
Not only did hwoarangmy what he said above, but he also:
- improved the code to avoid several memleaks by design, such as in rooms and render requests creation.
- Gathered the hard-coded creature prefix back into one sole location.
- Fixed the handling of goals between the server and the clients.
- Simplified some common logic about getting a player reference, and refreshing tile shapes.
- Simplified the whole level starting logic.
- Added optional timestamp support to log messages.
- Added a lot of missing server <-> client messages needed to make the game actually work.
- Probably even fixed a bug in the claiming logic.

Also, I wanted to send him special kudos for his ability to mimic the current code style guidelines and the overall code readability.

I have merged the pull request on github. I will have some free time on wednesday evening if all goes well, and I'll test all that more thoroughly.
And why not add some of my own contributions.

regards,

[Bertram]

Makes me realize. Let's challenge each other in some multiplayer game sometimes. ;D

[Danimal]

Thats a quite awesome update indeed . my only concern of what i read, is is alrigth a new turn is not sent to everyone if one of the clients doenst update? Isnt that avoidable lag?

Bertram and everyone, being able to play a multiplayer map is nearer and nearer, we only need to correct the lil buggers ai and some rooms

[Bertram]

Hi Danimal,

my only concern of what i read, is is alrigth a new turn is not sent to everyone if one of the clients doenst update? Isnt that avoidable lag?

This is a very relevant question.

Since hwoarangmy played on the same computer using two game instances, surely the lag almost didn't exist.
Dealing with lag is the very essence of it, even if we're lucky here, since OD is a semi-turn based game, making the lag compression needs much less proeminent.

If the question were on my shoulder only, I'd answer that I do think some hard/soft timeout should be set sometimes in the future.
To my eyes, this means for instance that a client can be "late" one or two turns max and that the server will wait a maximum of <insert-perfect-value-here> milliseconds before going to the next turn without every client leave.
Once a client is too far away, it is disconnected, and/or more severe measures are taken to make it catch up.

Also, at some point, creatures could get their position and current action along with their destination sent by the server, so that when a creature is lagging behind, its client make it catch up somehow. (By moving there faster, of teleporting if too far.)

All this may permit client to stay in sync when certain turns are very packets-heavy, for instance.

Of course, this is only my opinion, and the input of everyone here, especially hwoarangmy, will matter.

Regards,

P.S.: I'll crush you all on our first online game you all evil penguins! Mwahahaha, arg, errr, kof kof...

[hwoarangmy]

Makes me realize. Let's challenge each other in some multiplayer game sometimes. ;D

It would be nice to try a game through internet to see if it lags or not

is is alrigth a new turn is not sent to everyone if one of the clients doenst update? Isnt that avoidable lag?

I usually think it is better to wait a little bit at each turn than freeze during 1 second when a client is too late. I have tried to speed down the client and I've seen that at some point, the client was getting laggy. With the synchronisation, that didn't happen.

Dealing with lag is the very essence of it, even if we're lucky here, since OD is a semi-turn based game, making the lag compression needs much less proeminent.

If the question were on my shoulder only, I'd answer that I do think some hard/soft timeout should be set sometimes in the future.
To my eyes, this means for instance that a client can be "late" one or two turns max and that the server will wait a maximum of <insert-perfect-value-here> milliseconds before going to the next turn without every client leave.
Once a client is too far away, it is disconnected, and/or more severe measures are taken to make it catch up.

It is funny because I have the same thoughts but a different conclusion. I would rather say it is good to try a simple synchronisation and see if it works. If not, we will try to do better. As it is said, premature optimization is the root of all evil

If the question were on my shoulder only, I'd answer that I do think some hard/soft timeout should be set sometimes in the future.
To my eyes, this means for instance that a client can be "late" one or two turns max and that the server will wait a maximum of <insert-perfect-value-here> milliseconds before going to the next turn without every client leave.
Once a client is too far away, it is disconnected, and/or more severe measures are taken to make it catch up.

Also, at some point, creatures could get their position and current action along with their destination sent by the server, so that when a creature is lagging behind, its client make it catch up somehow. (By moving there faster, of teleporting if too far.)

I've read some stuff about optimizing network for games with prediction and stuff. The only thing I remember is that it would be nice to not have to deal with that

BTW, I make a mistake. I tried the game with windows so it would be nice to try with linux, not windows.
And at the moment, I am trying to go for the full server paradigm but I'm not sure I will be able to do it because the gamemap is much related to graphic stuff. We will see

[paul424]

I don't mind changign to github or other source hosting site, if it is true that it would attract more people . I don't mind even if things get more complex.
Also : so your point is to move away entirely from the SOURCEFORGE , right ? What I can say, for me before joining OD it was a way of quickly monitoring what happens in the world of open source games .... but for now we cannot exceed 100 downloads / week , so maybe it's not a big lost ...