Improvements to the ranking system

Improvements to the ranking system

Postby Alayan » 28 Oct 2019, 17:34

After a few months since 1.0 came out, it's easier to see some of the issues with the ranking system. Some were already known earlier, but I'm going to try and list everything:
- A high rated player will often get more points from a new player than from a mid-level player with a stabilized ranking. This creates a "noob farming" element at the top which is out of place.
- For players that only rarely disconnect, the points lost from a single accidental disconnect are typically too high. This is especially felt for players on the top.
- Players that disconnect very often get a deflated ranking that make them take points from the top and give them to the bottom. Romero2019 is extremely annoying because of this, he's a 2.5K rated player that can drive at 6K level but keeps quitting races. Beating him gives you close to nothing but losing to him loses you points. Besides completely blocking abusive quitters from some servers, the ranking system should be more robust.
- Many players only play ranked races against a limited number of other players, or have a huge proportion of their races against one or a few players. For any ranking system to work well, you need players to play a variety of opponents so that the "links" between players are strong enough. If A and B never played each other, their relative ranking will be much more accurate if you can link them through other players they played against. If you have pools of players that play together inside the pool but rarely will play with someone outside the pool, the ratings get distorted.
- A related issue is the possibility to cheat by farming points against a fake account that just disconnects (or race badly on purpose).
- A player can refuse to play any ranked race on tracks he dislikes, or against players he doesn't want to face.
- A player's ranking never "fades away" after long inactivity. On one hand, the latest available ranking is the best estimate we have of the player's strength, but on the other hand the more time passes without playing, and the more uncertain the estimate becomes. Thus, a "ranking uncertainty" estimate that goes up when inactive and down when playing races against similarly-skilled drivers (if the rating difference is big, the result of a race doesn't tell a lot about how big the difference should be) is needed.

Are there other issues ?
Alayan
STK Moderator
 
Posts: 220
Joined: 25 Mar 2012, 22:11

Re: Improvements to the ranking system

Postby kimden » 28 Oct 2019, 18:04

A player can refuse to play any ranked race on tracks he dislikes


I would add that while some official tracks are not present in Android version, ranked servers are forced to accept players with at least 70% of official tracks. So any player can actually remove 6 least favourite official tracks and still race on a ranked server. That doesn't only abuse the ranking system, imagine four such players with almost different absent sets, then they could block any ranked server.
User avatar
kimden
 
Posts: 8
Joined: 23 Jun 2019, 18:42

Re: Improvements to the ranking system

Postby Andet » 28 Oct 2019, 18:15

The only other thing I can think of is that the top scores don't seem to converge anywhere, so getting a really good rank requires that you play ~1,000 races and newcomers might be discouraged.
User avatar
Andet
 
Posts: 47
Joined: 08 Dec 2018, 01:40

Re: Improvements to the ranking system

Postby kimden » 28 Oct 2019, 21:47

I thought a bit and had some ideas.

1. For each player A whose points are recalculated, we can check if there is a 'suspicious' subset of his opponents in this game (e.g. if there is a player against whom A played at least B% of his ranked races, if there are two players against whom A played a least C% of ranked races and so on). If yes, the multiplier for the ranking change can be lowered. Though then we need to store all games count for each pair of players. Also I don't know if this actually works for a subset of rivals with ~16...30 players (or maybe this is ok?)

2. Similarly, we can check for a player if he/she played too much on a certain track, and if so, lower the change multiplier. Though I don't really know how to insert ranked addons in this system (and some rare picked tracks like Zen Garden or Abyss).

Of course, this shouldn't be done with players who has less than a certain amount of races. Also I guess the lowering value for multiplier shouldn't be constant (for example, 75% of Volcan Island vs 16% of six different tracks are different situations, but both 'suspicious').

About the frequently disconnecting players, maybe we can calculate their (and their rivals') changes based on their points without rage quits (which are greater than normal points, but also with other negative changes), but add changes to normal rating (for example, a player who has disconnected 16/16 times and obtained X < 2000.00 points will have rating changes later as if he had 2000.00 after 16 races, but instead of Y points (as without rage quits) he will get Y + (X - 2000.00)). Maybe this will help with reducing rage quits...

These ideas should be developed more (or thrown away) and are not final in any way.
User avatar
kimden
 
Posts: 8
Joined: 23 Jun 2019, 18:42

Re: Improvements to the ranking system

Postby eltomito » 29 Oct 2019, 22:28

What happens to me is that every once in a while, I have to quit in the middle of a race, because a cat jumps on my laptops and shuts the lid which screws me up, or something comes up which I have to deal with immediately, or there's a problem with my wifi that makes playing impossible.

I know these are all preventable issues (shut the cats in a box while playing, find a different girlfriend, use ethernet instead of wifi) but still, it would be nice if I was allowed to drop out of a race without losing any points, lets say, three times a week. Preferably, the system would be automatic, so that if the server detected I didn't finish a race in a reasonable time, it would pretend I never entered the race in the first place and it turn a blind eye towards my disastrous performance but no more than 3 times a week/a month or whatever you consider appropriate.

----

Regarding the fragmentation issue, I love racing the Brazilians but they're so far away that it's a bit hard to find a server where neither of us lags. Also, their time is very different, so we seldom meet online. I think this is something you can't really help. I'd love to race on the HK servers but Honk Kong is so far that I can never get decent ping there. So, the ranking system should take into consideration that a fairly strong degree of geographic fragmentation is inevitable.
eltomito
 
Posts: 131
Joined: 15 Mar 2013, 09:25

Re: Improvements to the ranking system

Postby Alayan » 30 Oct 2019, 01:39

eltomito {l Wrote}:What happens to me is that every once in a while, I have to quit in the middle of a race, because a cat jumps on my laptops and shuts the lid which screws me up, or something comes up which I have to deal with immediately, or there's a problem with my wifi that makes playing impossible.

I know these are all preventable issues (shut the cats in a box while playing, find a different girlfriend, use ethernet instead of wifi) but still, it would be nice if I was allowed to drop out of a race without losing any points, lets say, three times a week. Preferably, the system would be automatic, so that if the server detected I didn't finish a race in a reasonable time, it would pretend I never entered the race in the first place and it turn a blind eye towards my disastrous performance but no more than 3 times a week/a month or whatever you consider appropriate.


Giving "wlidcards" that allow a player to completely avoid losing points is not something I consider a great idea, as it incentivize quitting the race when you're doing particularly bad (a top player unusually losing to average players after some big mistakes, etc.). The incentive should rather be to keep on trying.

I know that sometimes it's something completely outside the fault of the player (some connection issue with the server during the race), but quits still need to give a loss imho.

However, the idea with a count of quitted races (and so a quit frequency) would be to make the punishment in rating lower when the player only rarely quit races. It would also be to tune the system so that frequent quitters have not much influence on the rating of other players, at least when they quit.

Regarding the fragmentation issue, I love racing the Brazilians but they're so far away that it's a bit hard to find a server where neither of us lags. Also, their time is very different, so we seldom meet online. I think this is something you can't really help. I'd love to race on the HK servers but Honk Kong is so far that I can never get decent ping there. So, the ranking system should take into consideration that a fairly strong degree of geographic fragmentation is inevitable.


I think this is not something that can be "fixed" either when it comes to people living far away. The fragmentation talk was mostly related to the fact that there aren't that many people on STK ranked servers so it's very possible to play many ranked races against mostly the same players. If I were to do 300 races in a row against Andet, our rankings would evolve to better show our relative strength to each other, but if our average rating was overevaluated or underevaluated, then the same will be true after those 300 races.
Alayan
STK Moderator
 
Posts: 220
Joined: 25 Mar 2012, 22:11

Re: Improvements to the ranking system

Postby QwertyChouskie » 30 Oct 2019, 17:15

Maybe maintain two ranking numbers internally: The user visible one, affected by quitting, and an internal one, not affected by quitting (or only slightly affected). The latter number would be used when calculating rating transfers, so that excellent players that regularly quit races have reasonable rating transfers when playing against others. (I remember Alayan recently mentioning a play that has a low rank but has the skills of a high-ranked player, which was causing rating transfers to be not accurate. This would theoretically solve that.)
Not an official dev, but a huge fan of STK!
User avatar
QwertyChouskie
 
Posts: 382
Joined: 29 Jun 2016, 14:57

Re: Improvements to the ranking system

Postby Alayan » 30 Oct 2019, 20:59

An illustration of the noob farming issue

6K+ players extracting 12 points from a new player that had quit the race.

Image
Alayan
STK Moderator
 
Posts: 220
Joined: 25 Mar 2012, 22:11

Re: Improvements to the ranking system

Postby benau » 31 Oct 2019, 05:01

There is a high ping workaround in git servers now, so as long as the internet is stable (no packet loss) you can just use the default value of max-ping and jitter tolerance.

Brazilian servers fix this!
Image
benau
STK Moderator
 
Posts: 153
Joined: 08 Dec 2015, 17:32

Re: Improvements to the ranking system

Postby relay » 02 Nov 2019, 15:46

Hi all,

first of all: thanks alot for working on and improving this beautiful game since years, we enjoy it alot with our kids and family.

I'm new in online playing it, and I feel discouraged to race against higher rank players. Not because lack of fun, but because of loosing points against top players.

Imho there is an easy to implement way to fix this:

- Solution part 1: If a player wins against a higher rank player, the points should be adapted stronger.
- Solution part 2: If a player wins against a lower rank player, the points should be adapted minimally only, if the lower rank player is maximum 25 ranks below him. (Winning against more than 25 ranks below oneself should not change ranking.)

- Why: If e.g. rank 50 player wins agains rank 100 player, he should never become a higher rank than 50, as he has not proved to race better than the ones in the list on top of him, and rank100 player should not go lower, as he was not proven to drive worse than the 101, 102, 103, ...

Thanks alot for the beautiful free game. :-)

Kind regards, and thanks for reading/reflecting
Sincerely, Relay
relay
 
Posts: 1
Joined: 02 Nov 2019, 15:22

Re: Improvements to the ranking system

Postby kimden » 03 Nov 2019, 17:20

I don't think cutting the rating change based on the rank difference (not points) makes ranking better. I played with many players and being 7th currently, it can be sometimes difficult for me to beat players in top 50 (not necessarily because my rank is too high for my skill, a player can ride some tracks exceptionally well). Probably for ranks 50-100 it's even more unusual, why shouldn't they get points for winning against players in top 200 whose ranks are only 800 less?
Also it would mean that to change rating, top players need always to play together which is unlikely. This can probably cause decrease of races for them.
User avatar
kimden
 
Posts: 8
Joined: 23 Jun 2019, 18:42

Re: Improvements to the ranking system

Postby kimden » 03 Nov 2019, 17:47

Also I think the current system doesn't encourage players to be faster in the way that 3 seconds advantage is equal to 20 seconds advantage in terms of points change (for a standard 2-minute race). So the leader cannot increase his points other way than slowing down his opponents (there are few ways like putting the gums everywhere or using switches). Don't know if this is intended though, but probably at least a small bonus should be given.

(Yes, sounds like 'give more to the richest', but the speed should be rewarded somehow.)
User avatar
kimden
 
Posts: 8
Joined: 23 Jun 2019, 18:42

Who is online

Users browsing this forum: No registered users and 1 guest