Jump to content
FORUMS
Sign in to follow this  
Stan

Performance-Based Matchmaking Explained (Interview)

Recommended Posts

P94FCDT.jpg

Khaldor is back with yet another interview with Lead Designer Travis McGeathy about performance-based matchmaking. This time around, Travis answered a lot of interesting questions and responded to player concerns about the new system.

Previous Performance-Based Matchmaking Coverage

Khaldor's Interview

  • The goal of the new system is to accelerate you where you should be for your MMR. You won't get massively punished if you had one bad game.
  • The new performance-based matchmaking system constantly tracks 20 different stats (criteria) every game. They are used to tell how a particular player in a particular situation is doing. Everything is being sliced up for that situation. It's the Hero you're playing, the battleground you're on, the game mode you're in, the region and then there are modifiers for things like game time.
  • Game duration won't have any impact on the system, prolonging the game won't have any effect.
  • The criteria are being measured for everybody, but how important they are is coming down to that particular situation. The system is measuring every game and looking at across the breadth of all the players what's the difference in the stats between a Bronze player and a Master player. There's more granularity in there, but it's looking to see what differentiates a lower skilled player and a higher skilled player and based on that it's weighing the 20 different factors in different ways.
  • The biggest factor is still winning or losing the game.
    • If you performed better, you're going to lose less points when your team loses a game.
    • If you, on theother hand, won the game, but your performance was not as good, you're going to gain less points.
  • Ult choice does not have any impact on MMR - it doesn't matter if you're playing a Varian and go full DPS or full tank. You're elevating some stats at the cost of others. The overall performance rating does not change a lot based on your ult choice, it's just about whether you're playing well overall.
  • If you're focusing on just 1 stat at the expenses of the other 19 stats, your overall performance will suffer. The system cannot be fooled to believe that you should be accelerated faster through the ranks when for example, you're trying to collect CC time as Kerrigan or Muradin.
  • Death time matters for all the stats.
  • Personal rank adjustment isn't going anywhere next Season, but they'd like to get rid of it in the future.
  • They'd like to make MMR visible in the future.
  • For the first period of time after a new Hero drops, performance-based matchmaking is shut off completely, until the machine learning system has gathered enough data to do that. The same applies to major Hero reworks or very specific scenarios (Cho'gall in Master League) - it uses the old system until there's enough data.
  • In case of any issues emerging with the new system, the engineers are able to shut the new matchmaking down entirely or for specific Heroes.
  • There may be loopholes that the developers will fix, but the new system effectively uses machine learning.
  • Players need to realize that by not doing anything (sitting at spawn) when a game is about to be lost will hurt them even more, because they will lose more points.
  • When handling feeders, the team is currently backed up by player reports and CS, but in the future, they'd like to have an automated system to evaluate feeding vs having a bad game.
  • Loss forgiveness is a new feature they'd like to roll out at some point to account for intentional feeders, leavers. The game either doesn't count as much for MMR or doesn't count at all for Ranked hits. 
  • The overall net effect is across the 20 different stats that are measured. If let's say you're in a double tank composition, your 1 stat (damage taken) may be lessened, because you are not the solo tank there, but your other stats will go up, because you've got more opportunities to do other things.
  • While inside vehicles (Dragon Knight, Garden Terror), your stats will go up really fast, but it's important to mention that the matchmaking compares your game to millions of other games on the same map and again, you sacrifice other stats in favor of the ones you gain in vehicles, so it's the same boucing act that balances itself out over time.
  • Revealing all the 20 criteria would cause poor game behavior, but the team works on a feature to show players areas they excelled in. No ETA for that.
  • Normal players don't need to be worry about the new matchmaking system. On the backend, it's making the matchmaker more accurate in the short period of time. It's okay just to worry about winning or losing. The overall game quality should increase, because players will be arriving faster at their MMR ranks.
  • The uncertainty boost in placement matches will not exist anymore in the new Season. Uncertainty is still there, but it's handled in a more dynamic way; it's based off of things like when you most recently played, streaks and so on.
  • Anybody who is nearby when a Mecenary camp is claimed will be granted credit toward the stat.
  • How they go about performance-based matchmaking in Heroes of the Storm and Overwatch is extremely different. 

Share this post


Link to post
Share on other sites

I have to say that I'm somewhat skeptical about how taking the dragon knight or garden terror wouldn't boost your in-game rating . Dragon Knight has CC, high damage AOE, and his main attack hits like a truck.

Share this post


Link to post
Share on other sites

I can only assume they will count your time in vehicle and compare it to people who had roughly same time in it.

I'm more interested about aspect that nubkeks said. At very high levels of play, you should be hiding in bush as a Warrior, effectively doing nothing, just making threat you are anywhere.

Share this post


Link to post
Share on other sites
12 hours ago, SleepySheepy said:

I'm more interested about aspect that nubkeks said. At very high levels of play, you should be hiding in bush as a Warrior, effectively doing nothing, just making threat you are anywhere.

Yeah, this and more. no matter what they do, there are a TON of in game situations where it's essentially impossible to measure how much impact a player is having. Even among "simple" stats like hero damage or damage absorbed, there can be a lot of difference in terms of the value of any given particular damage.

So, this is the trick: yes, all of these factors mean that performance based MMR will be less accurate than the current system (W/L only, over infinite matches, has perfect accuracy predicting how often you'll win or lose.) The trade-off, they hope, is speed. The problem with the current mmr system is that for it to represent player skill in any kind of accurate way you need hundreds and hundreds of games, which most players will just never get.

Overall, i'm optimistic. Machine learning has proved surprisingly effective at learning about these games in DOTA 2 (check out the recent series with OpenAI) so I think there's a good chance that this admittedly imperfect solution provides better results than the current "perfect but impractical" approach.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Stan
      Braxis Outpost, the single-lane Starcraft-themed battleground, is available as this week's brawl. It comes with standard ARAM rules, which include disabled Hearthstones and no healing at spawn. Complete 3 matches of Braxis Outpost to earn a Loot Chest.
      Blizzard (Source)
      This week’s Heroes Brawl is Braxis Outpost! A single-lane battleground set in the Koprulu Sector, it will be up to your team to survive in the lane and take down the enemy core, without the aid of Hearthstone or their Hall of Storms. Good luck, have fun!
      Rules:
      Shuffle Pick Choose from one of three randomly selected Heroes before entering the battle on Braxis Outpost. Be quick about it though, you only have 30 seconds to choose! ARAM Rules Players cannot use Hearthstone Players do not heal at their starting location. (Hall of Storms) Cores have no shields Mercenary Camps Braxis Outpost features two mirrored Goliath & Raven mercenary camps. Be the first team to take down the enemy Core to claim victory! Rewards:
      Complete three matches of Braxis Outpost to earn a Loot Chest! Find out more about the Heroes Brawl game mode on our Heroes Brawl site; and as always, you can find more information on this week’s Brawl by clicking the Brawl Info button at the bottom of the play screen when preparing to queue for the Brawl game mode.
    • By Stan
      Qhira now costs 10,000 Gold. This week's Heroic Deals include Junkrat, Kel'Thuzad, and The Lost Vikings.
      Click here to check out this week's Free-to-Play Hero rotation.
      Hero Sales
      Heroes Old Price New Price Junkrat 750 Gems 375 Gems Kel'Thuzad 750 Gems 375 Gems The Lost Vikings 750 Gems 375 Gems Skins
      Deathranger Nova Bundle can still be purchased for 1,500 Gems and includes:
      Deathranger Nova Mirage Deathranger Nova Rugged Deathranger Nova Ember Deathranger Nova Dusk Deathranger Nova
      Mounts
      3 mounts have been added to the game and they cost 10,000 Gold each.
      Scavenger's Ripper Outrider's Ripper Soldier's Ripper
    • By Stan
      This week's rotation includes Brightwing, Kel'Thuzad, Medivh, and more.
      Free-to-Play Hero Rotation: August 20, 2019
      The required player level remains unknown because Blizzard stopped updating the official forum post, so we're listing the Heroes alphabetically.
      Azmodan Brightwing Jaina Kel'Thuzad Medivh Muradin Nazeebo Raynor Rehgar Rexxar Samuro Sonya Uther Valeera (Source)
    • By Stan
      In Storm League, rank is always updated by taking the change in rating and determining the equivalent number of rank points to reward. While the Heroes team is quite happy with how it turned out, the system is causing problems in Bronze 5.
      In Bronze 5, players gain/lose far fewer points per game than players in other leagues and divisions. The underlying problem makes it harder to climb out of Bronze 5 and causes frustration and bad gameplay experience. The team is working on ways to improve it and provided an explanation of why this is happening.
      Blizzard (Source)
      With the launch of Storm League season 1 we have seen a dramatic improvement in rank distribution. One of the primary changes in this release was in how rank is determined. In the past, rank and rating were two entirely different concepts, both of which were updated differently after each game. It was very common for a player's rank and rating to diverge throughout each season which could cause the matchmaking system to struggle to find games.
      With the changes introduced this season, rank is always updated by taking the change in rating and determining the equivalent number of rank points to reward. This leaves very few ways rating and rank can diverge and results in most players being shown a number equivalent to their rating.
      Overall, we are very happy with how the changes are looking; however, we are aware of the inconsistencies players are currently experiencing in Bronze 5. With the new design, players in Bronze 5 will gain/lose far fewer points per game than players in all of the other leagues and divisions. The reason for this is a bit technical but we wanted to take the opportunity to elaborate on what's going on.
      Under the hood, matchmaking rating is simply a number on a bell curve. Every league and division encompasses a range of values on the same curve. For example, if the overall bell curve ranges from -10.0 to 10.0 and the range 1.0 to 1.5 maps to Gold 1, then a player with a rating of 1.25 will have the equivalent rank of Gold 1 & 500/1000 points. These numbers are intentionally made up as the real ones are always subject to change but the idea is the same.
      The reason for the decrease in points awarded in Bronze 5 is that the range of ratings that maps to Bronze 5 is much larger than other divisions. The difference is that this much larger range of ratings still maps to the same number of rank points. As a result, despite being rewarded the same change in matchmaking rating, the rank points awarded may be lower. This is visualized by the graph below:
      We understand that this inconsistency is a negative experience for players so we will be exploring some ways to improve it.
      Thanks for participating in Storm League and providing your feedback!
    • By Stan
      Industrial District is this week's brawl. The single-lane Overwatch-themed map comes with standard play and shuffle pick. The main goal is to destroy the enemy Core and playing three games rewards a Loot Chest.
      Blizzard (Source)
      This week’s Heroes Brawl is Industrial District! Slide around and slug it out in the heart of Volskaya in this Overwatch-themed, single-lane Battleground featuring lots of conveyor belts and tons of action.
      Rules:
      Shuffle pick - Choose from one of three Heroes before entering the battle. Standard play - no talent or level restrictions. The first team to destroy the enemy Core wins! Rewards:
      Complete three matches of the Industrial District Brawl to earn a Loot Chest! Find out more about the Heroes Brawl game mode on our Heroes Brawl site; and as always, you can find more information on this week’s Brawl by clicking the Brawl Info button at the bottom of the play screen when preparing to queue for the Brawl game mode.
×
×
  • Create New...