Official Explanation of Bungie's Matchmaking/Ranking System

Discussion in 'Halo and Forge Discussion' started by bnasty574, Jul 3, 2008.

  1. bnasty574

    bnasty574 Ancient
    Senior Member

    Messages:
    743
    Likes Received:
    0
    This is not mine, I got it from MLG, but I thought it was a very interesting read. I hope Bungie really makes the fixes, since this is something they could do relatively easy (read and you'll see). I know its a wall of text, just skim it if you must. If you don't care about the technicalities and just wanna know what you can do to rank up finally, skip to the second post.

    CREDIT
    __________________________________________________

    Nickel-9 posted a link to a BNet thread the other day in regards to Microsoft’s TrueSkill system. ( Bungie.net : Community : Forum Topic Listing : Forum Reply Listing ). I'm one of the two main contributors on that thread, and although no one knows everything about how the ranking system works, I would consider myself right up there with the most knowledgeable people on the subject. At the request of a few MLGers and for the benefit of people leveling up accounts for money, I am going to expand upon the BNet thread that was posted here the other day.

    Since the time of the original posting, we have found out many new quirks about the ranking system, so I will try and summarize how things work the best I can for you. I want to give credit to Jay120171 (z RaZoR z ), who has contributed to understanding the system more than anyone else I know, as well as the OP El Kafungus, who I don’t think plays any more but used to post here on MLG. I will blatantly plagiarize a few things Jay has written so I can post this as efficiently as possible.

    Bungie uses Microsoft's TrueSkill System. This system has been applied against a boatload of other XBOX games and is a pretty plain vanilla system. There are some MINOR tweaks each game creator can adjust to better fit their game...Bungie themselves have made a few minor tweaks to the rank system. You can read about TrueSkill here Microsoft Research . I also suggest reading Bungie's internal Power Point presentation that explains their own adjustments to the system http://www.bungie.net/images/Inside/...atchmaking.ppt . I also suggest a newer but virtually unread post by Jay http://www.bungie.net/Forums/posts.a...1-p=1#19254454 .

    In its most dumbed down form, TrueSkill's main purpose is to provide quality matchmaking to players...the actual side effect is to give players a visible rank.
    The skill displayed in each of your playlists consists of the following equation:

    Displayed Skill = MU - (K * Sigma)
    ------ Or the anti-math version -----

    Displayed Skill = Conservative Skill - (Degree of Conservativeness * Degree of Uncertainty)

    Everyone is ranked 1-50 and the curve has a somewhat normal distribution centered at 25 in theory, but more like 20 in reality. If you remember your statistics courses (for those that have taken them), this means people in the 42 + range, although the norm here in MLG, are actually statistically quite rare. Although I am a bit lazy and have not tried to calculate the value of each standard deviation, I think you can safely predict that anyone over about 42 is probably in the top 4.6% of the population, and anyone that is a 50 is likely in the top 0.3%. The reason the mean is centered closer to 20 in reality is because of the # of new players that only play a few ranked games and then never play again.

    Everyone starts off in a playlist with a rank of 1 and a high Sigma value…this Sigma value is one of the things Bungioe is allowed to change. Your rank in one playlist is completely independent of your rank in any other playlist. I.E., You have a separate MU and Sigma for each playlist. For those of you that have taken statistics courses, Sigma is not used the same way it is in a pure volatility sense. Instead, Sigma drops after virtually every game you play, more so when you lose rather than when you win. Bungie adds a very small amount back to Sigma after every game to make sure it doesn't eventually go to 0, because then the math would stop working and all rank progressions would stall. In some very rare instances, if you can go on a very long winning streak, we have found evidence you can slightly increase your Sigma. What does this mean to you? It means once you start to lose games, your rank progression is going to go a lot slower...almost permanently. Once you have lost enough games, you are looking at 10+ net wins to move a rank in any team playlist, and probably about 4-6 wins in LW's. You all know when you first start a playlist you can move 1-3 ranks every win. The basic rule of thumb is that once you have lost about 20-30 games in a playlist, you will start moving extremely slow.

    So, we have determined the best way to rank up is to win. However, winning does not guarantee you will rank up (gain MU). Each game you complete, the system evaluates you against each and every other player you played against. However, not in the way you think. Before the game starts, the system makes a prediction on whether you should win or lose, based on everyone else's MU and Sigma values. If the system expected you to win and you did, you will not gain much MU. Lose when you were expected to, you won’t lose much. Win when you were expected to lose, you will gain a significant amount of MU. The ranking system DOES NOT take into account Kill to Death Ratio, MVP, Medals, Headshots etc. It ONLY evaluates whether you OR your team performed according to expectations with a win or a loss. Anyone that tells you any different is 100% wrong...this has been confirmed by both Bungie and Microsoft. EXP is also generally irrelevant to your ranking ability, although some people claim they have found a way to boost accounts when have 0 EXP (more on this later).

    The system can determine your skill in LW’s and Doubles games a lot quicker than larger team games. I fact in LW’s the system’s creators think they can find your skill level in as little as 5-10 games or so. In theory, although not necessarily in practice, it is a lot quicker to rank up in these two playlists. Ranked BTB and Team Slayer/MLG/SWAT in theory should take longer. In practice of course we know LW’s gets really tough in the upper 40’s, and is also a game more dictated by spawns and killing speed, something TrueSkill cannot incorporate. Doubles seems to be the best bet for quick leveling. On average, a true level 50 player will take an average of 75-150 games to reach that rank. There are some quicker methods, which I will touch upon later in this thread. I think the quickest to 50 using legit methods I know of was in about 75 games in LW’s.

    Another drawback to the system is the matchmaking itself in the high 40’s. Many of you will find you have won 20+ games in a row and have not ranked up from say 48 to 49. This is purely due to poor matchmaking quality. The higher the rank you have, the less of a pool of people to match you against. You will undoubtedly have 50% of your games against people below your rank. When you beat them, because the system expected you to beat them, you will gain very little if any rank.
     
    #1 bnasty574, Jul 3, 2008
    Last edited: Jul 3, 2008
  2. bnasty574

    bnasty574 Ancient
    Senior Member

    Messages:
    743
    Likes Received:
    0
    Things that can cause delays in ranking up:

    1) Playing with the same teammates: There are many known cases of people’s ranks being slowed considerably by playing with the same people. In fact, there are some Team Slayer cases of 4 guys, all legit 50’s, that can’t get past rank 16-18…all because they are undefeated. The TrueSkill system, when playing team games with the same people, cannot differentiate between your skill and that of your teammates. Basically until you split from them, everyone’s Sigma stays so high the ranking stalls out. This is a mathematical error basically in the way the equation calculates stuff. However, it’s also an error they likely won’t fix, because the instances of them are so rare.

    2) Never losing: If you never lose, your Sigma will either remain at the initial high level or possibly even have a slight increase. If it just stays high this is fine…you will rank quickly. However, if it increases from that already high level you have the same mathematical problem as 1). You will find you will rank fine from around level 1-20, but after that you should really make sure you lose a game or two so that the formulas don’t bugger up. I know this sounds ridiculous, but it is true in the cases we have looked at.

    3) Mixed Parties: Playing in a mixed party typically slows the rank progression on the higher ranked player, yet increases the progression of the lower ranked player. A party is considered mixed when skill levels are different by more than 10. It is true that when starting a new account, playing with a team of guys ranked way higher than you will give you a huge initial boost, ranking you 2-4 ranks for every win. However I caution that you will permanently damage the higher ranked players’ Sigma values as you continue to team up…and eventually yours as well. Bungie relaxed the penalty it has previously applied to mixed parties a little, but it still does macro damage to ranking.

    Other stuff:
    1) Microsoft is currently looking into making Sigma more variable...more like the statistical definition of Sigma. Basically there have been many complaints of people starting second accounts because their ranks are stuck. They are looking into giving some Sigma increases for winning streaks, so that players that have improved but were initially garbage at the game can now achieve their new skill level without playing 1000+ games to get there.

    2) Apparently some people have found ways to boost accounts to 50 in less than 35 games. This is alluded to in Bungie.net : Community : Forum Topic Listing : Forum Reply Listing and Bungie.net : Community : Forum Topic Listing : Forum Reply Listing . I don’t really want to discuss it in depth as people are going to get in trouble for it. I might come back later once the research is over to discuss it more fully. The basic gist is that people have broken apart the calculation and found a way, through guys repeatedly quitting, to raise sigma values to high levels and help another player rank quicker. I DO NOT ADVISE YOU PARTICIPATE IN THIS KIND OF BOOSTING. Not so much because I care, but because people ARE going to get banned for it.

    3) Should you start a second account to rank up quicker?: Tough question. If you intend to play say 5,000-10,000 games of Halo 3 over time, you will eventually reach your TrueSkill no matter how low your Sigma is. However, for some people that have improved a lot over the last 9 months, a new account make get them to their rank in 150 games vs. the extra 500-1,000 their main account may take. This is the one true flaw of TrueSkill…it takes a long time to recognize a skill change for a player that has improved and places zero utility on people’s free time. All personal preference really though…there are definitely a lot of cases where a new account would be appropriate to save time. Most importantly though, if someone is paying you to rank them up to 50, there is a good chance it will get done quicker on a new account. I think the rule of thumb would be if they are anything more than a Grade 2, they probably have really low Sigma. Take a look through their games and get a good sense on how much they have lost.

    So what is my suggested method for ranking up to 50 the quickest without risking a ban for boosting? Although this is more hypotheses based on many facts we have learned over 9 months, rather than a for sure thing, I suggest:
    1) Play doubles
    2) Play only with partners you know are 50’s
    3) Switch your partners about every 5 games
    4) Avoid mixed parties other than maybe at the very beginning of a new account
    5) Make sure to lose a couple games once you have hit about rank 15-20
    6) Use the “Closest Skill” feature to search for games as you approach the high 30’s/low 40’s, as matchmaking quality will deteriorate greatly at those levels

    Well that’s all for now. I’ll update stuff whenever we find out more from testing or if Microsoft makes some of the tweaks they have been looking at. I will also have Jay possibly take a look at this to see if he can add anything or find any inadvertent mistakes.
     
    AllseeingEntity likes this.
  3. AllseeingEntity

    AllseeingEntity Ancient
    Senior Member

    Messages:
    250
    Likes Received:
    0
    very informative. i wonder how many people shall come across this and not understand...?
     
  4. bnasty574

    bnasty574 Ancient
    Senior Member

    Messages:
    743
    Likes Received:
    0
    I'd turn down the wall of text thing, but it is a very well formed hypothesis
     
  5. Playerhata27

    Playerhata27 Ancient
    Senior Member

    Messages:
    2,483
    Likes Received:
    6
    Very informative.But do you expect me to read all of it??
     
  6. bnasty574

    bnasty574 Ancient
    Senior Member

    Messages:
    743
    Likes Received:
    0
    I'm not your ****ing teacher do whatever you ****ing please
     

Share This Page