Stan

Performance-Based Matchmaking Explained (Interview)

Sign in to follow this  

4 posts in this topic

P94FCDT.jpg

Khaldor is back with yet another interview with Lead Designer Travis McGeathy about performance-based matchmaking. This time around, Travis answered a lot of interesting questions and responded to player concerns about the new system.

Previous Performance-Based Matchmaking Coverage

Khaldor's Interview

  • The goal of the new system is to accelerate you where you should be for your MMR. You won't get massively punished if you had one bad game.
  • The new performance-based matchmaking system constantly tracks 20 different stats (criteria) every game. They are used to tell how a particular player in a particular situation is doing. Everything is being sliced up for that situation. It's the Hero you're playing, the battleground you're on, the game mode you're in, the region and then there are modifiers for things like game time.
  • Game duration won't have any impact on the system, prolonging the game won't have any effect.
  • The criteria are being measured for everybody, but how important they are is coming down to that particular situation. The system is measuring every game and looking at across the breadth of all the players what's the difference in the stats between a Bronze player and a Master player. There's more granularity in there, but it's looking to see what differentiates a lower skilled player and a higher skilled player and based on that it's weighing the 20 different factors in different ways.
  • The biggest factor is still winning or losing the game.
    • If you performed better, you're going to lose less points when your team loses a game.
    • If you, on theother hand, won the game, but your performance was not as good, you're going to gain less points.
  • Ult choice does not have any impact on MMR - it doesn't matter if you're playing a Varian and go full DPS or full tank. You're elevating some stats at the cost of others. The overall performance rating does not change a lot based on your ult choice, it's just about whether you're playing well overall.
  • If you're focusing on just 1 stat at the expenses of the other 19 stats, your overall performance will suffer. The system cannot be fooled to believe that you should be accelerated faster through the ranks when for example, you're trying to collect CC time as Kerrigan or Muradin.
  • Death time matters for all the stats.
  • Personal rank adjustment isn't going anywhere next Season, but they'd like to get rid of it in the future.
  • They'd like to make MMR visible in the future.
  • For the first period of time after a new Hero drops, performance-based matchmaking is shut off completely, until the machine learning system has gathered enough data to do that. The same applies to major Hero reworks or very specific scenarios (Cho'gall in Master League) - it uses the old system until there's enough data.
  • In case of any issues emerging with the new system, the engineers are able to shut the new matchmaking down entirely or for specific Heroes.
  • There may be loopholes that the developers will fix, but the new system effectively uses machine learning.
  • Players need to realize that by not doing anything (sitting at spawn) when a game is about to be lost will hurt them even more, because they will lose more points.
  • When handling feeders, the team is currently backed up by player reports and CS, but in the future, they'd like to have an automated system to evaluate feeding vs having a bad game.
  • Loss forgiveness is a new feature they'd like to roll out at some point to account for intentional feeders, leavers. The game either doesn't count as much for MMR or doesn't count at all for Ranked hits. 
  • The overall net effect is across the 20 different stats that are measured. If let's say you're in a double tank composition, your 1 stat (damage taken) may be lessened, because you are not the solo tank there, but your other stats will go up, because you've got more opportunities to do other things.
  • While inside vehicles (Dragon Knight, Garden Terror), your stats will go up really fast, but it's important to mention that the matchmaking compares your game to millions of other games on the same map and again, you sacrifice other stats in favor of the ones you gain in vehicles, so it's the same boucing act that balances itself out over time.
  • Revealing all the 20 criteria would cause poor game behavior, but the team works on a feature to show players areas they excelled in. No ETA for that.
  • Normal players don't need to be worry about the new matchmaking system. On the backend, it's making the matchmaker more accurate in the short period of time. It's okay just to worry about winning or losing. The overall game quality should increase, because players will be arriving faster at their MMR ranks.
  • The uncertainty boost in placement matches will not exist anymore in the new Season. Uncertainty is still there, but it's handled in a more dynamic way; it's based off of things like when you most recently played, streaks and so on.
  • Anybody who is nearby when a Mecenary camp is claimed will be granted credit toward the stat.
  • How they go about performance-based matchmaking in Heroes of the Storm and Overwatch is extremely different. 

Share this post


Link to post
Share on other sites

I have to say that I'm somewhat skeptical about how taking the dragon knight or garden terror wouldn't boost your in-game rating . Dragon Knight has CC, high damage AOE, and his main attack hits like a truck.

Share this post


Link to post
Share on other sites

I can only assume they will count your time in vehicle and compare it to people who had roughly same time in it.

I'm more interested about aspect that nubkeks said. At very high levels of play, you should be hiding in bush as a Warrior, effectively doing nothing, just making threat you are anywhere.

Share this post


Link to post
Share on other sites
12 hours ago, SleepySheepy said:

I'm more interested about aspect that nubkeks said. At very high levels of play, you should be hiding in bush as a Warrior, effectively doing nothing, just making threat you are anywhere.

Yeah, this and more. no matter what they do, there are a TON of in game situations where it's essentially impossible to measure how much impact a player is having. Even among "simple" stats like hero damage or damage absorbed, there can be a lot of difference in terms of the value of any given particular damage.

So, this is the trick: yes, all of these factors mean that performance based MMR will be less accurate than the current system (W/L only, over infinite matches, has perfect accuracy predicting how often you'll win or lose.) The trade-off, they hope, is speed. The problem with the current mmr system is that for it to represent player skill in any kind of accurate way you need hundreds and hundreds of games, which most players will just never get.

Overall, i'm optimistic. Machine learning has proved surprisingly effective at learning about these games in DOTA 2 (check out the recent series with OpenAI) so I think there's a good chance that this admittedly imperfect solution provides better results than the current "perfect but impractical" approach.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Stan

      Free Hero rotation for the week of December 12 is here.
      Blizzard (Source)
      Free-to-Play Hero Rotation: December 12, 2017
      Li Li ETC Zagara Butcher Kael'thas Artanis Li-Ming Brightwing Varian Malthael Stitches (Slot unlocked at Player Level 5) Gul'dan (Slot unlocked at Player Level 10) Stukov (Slot unlocked at Player Level 15) Sgt. Hammer (Slot unlocked at Player Level 20)
    • By Stan

      Heroes of the Storm Highlights are back with the 97th episode of WTF Moments.
      Don't forget to submit your own replays here for a chance to be featured in an upcoming episode!
      Previous Episodes
      WTF Moments Episode 96 WTF Moments Episode 95 WTF Moments Episode 94
    • By Stan

      The first ranked season of 2018 starts during the week of December 12 and there's a new set of rewards that you can earn by playing ranked games in the coming months.
      With that said, Hanzo should go live on December 12 as well.
      Blizzard (Source)
      Season Roll Dates
      2018 Season 1 will begin as soon as our next patch is available for download in each region throughout next week. Check the start and end dates below and make sure you’re able to dive into the rankings as soon as the season rolls.
      EVENT DATE 2018 Season 1 Start Week of December 12, 2017* 2018 Season 1 End Week of March 6, 2018* *Exact date may vary slightly by region.
      Ranked Play Updates
      Along with the start of 2018 Ranked Season 1, we’re introducing performance-based matchmaking to all matchmaking modes in Heroes of the Storm.
      Performance-based matchmaking will help our system better evaluate individual players' in-game performance during a match, and adjust how much their rank and matchmaking ratings (MMR) increase or decrease at the end of that game. If players perform beyond expectations for their current MMR, the system will award more when they win, and subtract less when they lose. As a result, players will reach ratings that accurately match their skill level more quickly than they did in our previous system.
      There's more to the matchmaking improvements we're introducing, such as how the system differentiates highly-skilled play from one hero to the next. Be sure to read our in-depth blog on performance-based matchmaking to catch all the details before you jump into ranked play—or any matchmaking mode—next season.
      2018 Season 1 Rewards
      Players who brawl their way through the leagues during the upcoming season can claim new portraits, mounts, and gold rewards for their achievements in Hero and Team League. Take a look at everything that’s up for grabs this season.
      Hero League Portraits
      Team League Portraits
      Mount Rewards

      HERO LEAGUE RANK REWARDS Bronze 500 Gold Bronze Hero League Portrait Silver 750 Gold Silver Hero League Portrait Gold 1000 Gold Gold Hero League Portrait Platinum 1250 Gold Season 1 Mount Platinum Hero League Portrait Diamond 1500 Gold Season 1 Mount Diamond Hero League Portrait Master 1750 Gold Season 1 Mount Epic Season 1 Mount Master Hero League Portrait Grand Master 2000 Gold Season 1 Mount Epic Season 1 Mount Grand Master Hero League Portrait TEAM LEAGUE RANK REWARDS Bronze 500 Gold Season 1 Mount Bronze Team League Portrait Silver 750 Gold Season 1 Mount Silver Team League Portrait Gold 1000 Gold Season 1 Mount Gold Team League Portrait Platinum 1250 Gold Season 1 Mount Platinum Team League Portrait Diamond 1500 Gold Season 1 Mount Diamond Team League Portrait Master 1750 Gold Season 1 Mount Epic Season 1 Mount Master Team League Portrait Grand Master 2000 Gold Season 1 Mount Epic Season 1 Mount Grand Master Team League Portrait In addition to everything we’ve mentioned so far, there will be 10 placement matches waiting for you on the other side of the ranked season roll. Once you complete them in Hero or Team League, you’ll receive an initial ranking that is based on your performance during placements and your rank from the previous season.
      If you’re ready to begin your Ranked Play journey in Heroes of the Storm but are unsure how, head over to our Ranked Play Guide to get started on the right foot. 
      Good luck, and have fun during the first ranked season of 2018!
    • By Stan
       

      Blizzard just updated next week's sales. Get Johanna for 250 Gems & Ragnaros for 375.
      Blizzard (Source)
      Weekly Collection Update: Dec 12 – 19, 2017
      Next week's Collection Update will begin on Tuesday, December 12! Check out the new list of featured items and Hero discounts below.  Upcoming Hero Sales
      Johanna — Sale Price: 250 Gems Ragnaros — Sale Price: 375 Gems The following Skins and Mounts will enter the Featured Item Rotation, and will be unlockable using Gems until next week's Collection Update:
      Upcoming Featured Skins
      Scarlet Johanna Sulfuron Ragnaros Midnight Arcanist Jaina Green Eagle Eye Tyrande Harvest Halequin Nazeebo Bronze Tiger Kharazim Angelic Valla Infested Tychus Upcoming Featured Mounts
      Real Dream Championship Banner Shadowmoon Hellboar Weekly Sale Bundle
      If you’d rather pick up everything at once, we’re also going to add a new Weekly Sale Bundle to the Featured tab in your Collection. This limited-time bundle contains all of the items above at a discounted Gem price, but will only be available until next week’s Collection Update takes place.
    • By Stan

      After two weeks of Escape from Braxis it's time for Garden Arena!
      Blizzard (Source)
      This week's Heroes Brawl mode is Garden Arena: a best-of-three Arena match featuring the Garden Terror and its Shamblers. Gather seeds and overrun the enemies’ core with the power of nature!

      Rules:
      Choose from one of three randomly selected Heroes before entering the Arena. Be quick about it though, you only have 30 seconds to choose! Everyone will begin the round at level 10 and will be asked to choose a Heroic Ability. No other talents will be available. During each round, slay the enemy team’s Heroes and capture Seeds from the Garden Terror and its Shamblers. The first team to gather 100 Seeds wins the round! The first team to win two rounds will claim victory! Rewards:
      Complete three matches of Garden Arena to earn a Loot Chest! Find out more about the Heroes Brawl game mode on our Heroes Brawl site; and as always, you can find more information on this week’s Brawl by clicking the Brawl Info button at the bottom of the play screen when preparing to queue for this exciting new game mode.