If you've ever played online multiplayer games like Halo on the Xbox console, you'll know that being matched up with the right teammates and opponents is a critical part of the experience. Getting matched with an opponent that's way above your skill level isn't just demoralizing for you; it's also a waste of time for them (you generally don't advance in the game unless you beat a player that's of a higher level than you). The problem is even more complex for team games: there's nothing worse than being in a team with a bunch of newbies and then being pwned by a bunch of veterans on the other side.
Microsoft sets up online players into the best games for their skill level using the TrueSkill system, which is a Bayesian generalization of the Elo system used in chess matches. The basic methodology is described in this 2007 Microsoft Research paper, which used C# and F# for the data analysis and R for the visualizations. In the chart below, we see how the TrueSkill system performs compared to the Elo system for various kinds of games. An ideal matching system would generate the most number of drawn (tied) games, on the basis that players of equal skill are most likely to tie. According to the chart, the TrueSkill system works best for "Free for All" (e.g. Halo "Slayer") and "Head-to-Head" (one-on-one) matches, but not so for "Small Team" matches (which according to the paper are mostly capture-the-flag games).
The methodology has certainly progressed since 2007, and the new XBox One console features an all-new ranking system. Still, it's interesting to see the foundations of the system that matches players in Microsoft games. Thanks to reader Mikhail Popov for the tip, and to Microsoft's Tom Minka for some additional background on the paper.
Microsoft Research: TrueSkill(TM): A Bayesian Skill Rating System
Comments