How to write a lobby server

[Cardinal]

Hi guys,

So I'm writing a Chess matchmaking system based on a Lobby view with gaming rooms, general chat etc. So far I have a working prototype but I have big doubts regarding some things I did with the server. Writing a gaming lobby server is a new programming experience to me and so I don't have a clear nor precise programming model for it. I also couldn't find a paper that describes how it should work. I ordered "Java Network Programming 3rd edition" from Amazon and still waiting for shipment, hopefully I'll find some useful examples/information in this book.

Meanwhile, I'd like to gather your opinions and see how you would handle some things so I can learn how to write a server correctly. Here are a few questions off the top of my head: (may be more will come)

First, let's define what a server does. It's primary functionality is to hold TCP connections with clients, listen to the events they generate and dispatch them to the other players. But is there more to it than that?

Should I use one thread per client? If so, 300 clients = 300 threads. Isn't that too much? What hardware is needed to support that? And how much bandwidth does a lobby consume then approx?

What kind of data structure should be used to hold the clients' sockets? How do you protect it from concurrent modification (eg. a player enters or exists the lobby) when iterating through it to dispatch an event without hurting throughput? Is ConcurrentHashMap the correct answer here, or are there some techniques I should know?

When a user enters the lobby, what mechanism would you use to transfer the state of the lobby to him? And while this is happening, where do the other events bubble up?

Input is greatly appreciated. Thanks!

Screenshot : http://goo.gl/pYqM3

[charlie]

Don't make assumptions. 300 threads is only a lot if they are all actively requesting the CPU. In a network server, many threads will be in a waiting state as they are listening for a response from the client. Few will be active at any one point in time.

[Knitter]

Meanwhile, I'd like to gather your opinions and see how you would handle some things so I can learn how to write a server correctly. Here are a few questions off the top of my head: (may be more will come)

The best way to answer all these would be to make you implement and test it

First, let's define what a server does. It's primary functionality is to hold TCP connections with clients, listen to the events they generate and dispatch them to the other players. But is there more to it than that?

It really depends on how closely you want to look at the server's features. Generally we could say that yes, the server does nothing more than manage the connections to players, but it can also be seen as a login and authorization gateway and many other tasks. It depends on what you want your server to do.

Should I use one thread per client? If so, 300 clients = 300 threads. Isn't that too much? What hardware is needed to support that? And how much bandwidth does a lobby consume then approx?

Since you mentioned Java, I assume you're working with that programming language, you can use some of the classes Java offers you to create optimized threads and use thread pools. Even if you use normal thread classes, you can implement a simple thread pool. Thread pools will offer a bit more performance when creating new connections. 300 threads is not much, I just booted the computer and have only the browser working and already have about 150 threads running around, to a server, 300 threads/connections is not that much and many will be just waiting new input. Naturally, this kind of server model will have it's limitations, you can't just keep on creating threads .

Bandwidth will depend heavily on your protocol (binary, text based, with/without indentation, JSON, etc) but for the lobby I wouldn't worry too much about it.

What kind of data structure should be used to hold the clients' sockets? How do you protect it from concurrent modification (eg. a player enters or exists the lobby) when iterating through it to dispatch an event without hurting throughput? Is ConcurrentHashMap the correct answer here, or are there some techniques I should know?

Talking about performance before having a performance problem is not very good, you're just trying to optimize before even having some code. Still, use the structure that makes sense to you, what is easier use when inserted in the rest of the code. I wouldn't say that a ConcurrentHashMap was a solution since you don't have a concurrency problem yet and there are several ways to prevent one. The usual synchronization can be applied on the methods that change the data structure, or you can use a simple copy technique, where you always iterate a copy of the structure or you can iterate the structure normally and you put the new/removed elements in a temporary data structure and do the changing instructions after iterating the main list.

When a user enters the lobby, what mechanism would you use to transfer the state of the lobby to him? And while this is happening, where do the other events bubble up?

Again, depends on what is your protocol and what is your lobby. Will a lobby for a user be nothing more than a list of connected users and the last 15 messages? Will it have more information? It's hard to answer with just what you posted.

[smcameron]

I had a start on a generic lobby server (in C) here:
https://github.com/smcameron/ssgl

However, I got stalled on NAT punchthrough.

See: http://www.mindcontrol.org/~hplus/nat-punch.html

There's a thread about it on here somewhere...

Oh, here it is.

-- steve

[Cardinal]

Thank you so much for your responses. Let me clear something up. Right now I have a fully functional system. The only problem is that now it works with a few players but it will not scale, or rather, I don't know if it will. I think my real questions should be "What are the good design practices to write a scalable server?" For example, the server maintains a map with player id as key and player socket as value. When a player generates an event (eg writes something in general chat) the server dispatches this to the other players by iterating through the map and locking it to prevent concurrent modification. Let's say there are 300 players online generating various events, the threads on the server will contend too much for the map lock and throughput and service time will suffer. Later, I have come to learn that there's a map implementation designed for concurrency, ConcurrentHashMap, but I don't know how good it really is and whether replacing my map instance with a ConcurrentHashMap will be sufficient to support 300 players.

There's also the thread per client problem. I am not sure about this design, I worry 300 threads is too much. There seems to be another model where the server doesn't keep one thread per active connection but instead keeps a list of the connections and then have a single thread poll the connections periodically to see if there are events at the TCP layer pending to be dispatched and notify some listener.This would cure the server from too many threads but at the cost of lower service time. (Or so it seems to me)

I could probably figure out how to write the server well if I had a means to test it under moderate and heavy load (200 - 300 connections in one lobby) but I would need actual users. Large organizations have the resources and the people to run those kind of tests. But how would a sole developer do that?

I think I will write some kind of bot, populate the server with many instances, and see how it behaves. I don't know if I am being naive but I'll try and see how far I get.

[Knitter]

The ConcurrentHashMap is heavier since it will deal will synchronization in methods where you may not need it. Removing clients/adding clients is easily made safe by using a temporary list or map to hold the references or by synchronizing only a small set of the operations that have problems with concurrency.

Why don't you just test what you have? A simple automatic system, either as a script or a Java application will allow you to simulate several clients, naturally you'll need one machine for the server and others for the clients but it may provide some help. Also, the question about scalability will heavily depend on how many clients per server you want to have, you can divide de work load between other servers or clients that behave as servers.

Whatever choice you make, I believe you're thinking too much about a problem you don't yet have .

[Cardinal]

Actually ConcurrentHashMap ensures thread safety by using very fine grained locks. Instead of using a single global lock for the backing array, it uses multiple ones. Each lock guards a subset of the hash buckets. This allows multiple threads to work with different portions of the map concurrently. Only when two threads need to access the same portion of the map that mutual exclusion happens. It is said to have very good performance.

[sireus]

From what I understood, I think it doesn't make sense to use threads at all in your implementation. If you 1) iteratively step through the list of connected players and 2) lock everything whenever actual action is performed, then they're just useless. In a server, threads only make sense (not much even there, though) for things like HTTP servers where the connected clients have no relation to each other. Otherwise, a threaded model only adds complication.
If you could tell more exactly what the server should actually do, it would be easier to give a more reliable answer ;)

[smcameron]

Thank you so much for your responses. Let me clear something up. Right now I have a fully functional system.

Did you implement NAT punch through? If so I'd be curious to see your implementation.

-- steve

[Cardinal]

Thank you so much for your responses. Let me clear something up. Right now I have a fully functional system.

Did you implement NAT punch through? If so I'd be curious to see your implementation.

-- steve

I don't have the NAT traversal problem here because there's no direct communication between players. Everything is relayed by the server. However, I once wrote a peer to peer chat application and used a technique called UDP hole punching. Basically, you do an infinite loop during witch the participants keep throwing UDP packets at each other until they create holes in their NATs since at some point the router will look at an incoming packet and see that the headers match an earlier outgoing packet and thus will not drop it. That's when a connection is established.

From what I understood, I think it doesn't make sense to use threads at all in your implementation. If you 1) iteratively step through the list of connected players and 2) lock everything whenever actual action is performed, then they're just useless. In a server, threads only make sense (not much even there, though) for things like HTTP servers where the connected clients have no relation to each other. Otherwise, a threaded model only adds complication.
If you could tell more exactly what the server should actually do, it would be easier to give a more reliable answer

Actually, there is a debate between the thread per client model and the event driven model. (see http://capriccio.cs.berkeley.edu/pubs/t ... s-2003.pdf)
I just don't know what is good for me. For an example of what I'm trying to do, visit the gaming lobbies at Voobly.com. It's exactly what I want to build.

[charlie]

I would recommend asking this on stackoverflow.com or programmers.stackexchange.com (not that I wish to discourage discussion here).

[Cardinal]

I'm asking everywhere ^^

[FreakNigh]

A lobby server isn't really any more complicated then a chat server. The interesting part is how do you initiate the game? If you have individual clients starting the games on their own machines then they need to have ports forwarded which is normally too complicated for the average user.

Also I recommend always avoiding threads. They should only be used when you can't avoid it ie you have to call some other libraries load file function and you don't want it freezing your app. Also some servers have thread / processes per user limits.

[Cardinal]

A lobby server isn't really any more complicated then a chat server.

Thank you for bringing up this point. Chess moves are just text after all. So I could probably find some inspiration by searching for "Java chat server".

The interesting part is how do you initiate the game? If you have individual clients starting the games on their own machines then they need to have ports forwarded which is normally too complicated for the average user.

Chat rooms have Chess boards embedded in them.