Stan

Performance-Based Matchmaking Explained (Interview)

Sign in to follow this  

4 posts in this topic

P94FCDT.jpg

Khaldor is back with yet another interview with Lead Designer Travis McGeathy about performance-based matchmaking. This time around, Travis answered a lot of interesting questions and responded to player concerns about the new system.

Previous Performance-Based Matchmaking Coverage

Khaldor's Interview

  • The goal of the new system is to accelerate you where you should be for your MMR. You won't get massively punished if you had one bad game.
  • The new performance-based matchmaking system constantly tracks 20 different stats (criteria) every game. They are used to tell how a particular player in a particular situation is doing. Everything is being sliced up for that situation. It's the Hero you're playing, the battleground you're on, the game mode you're in, the region and then there are modifiers for things like game time.
  • Game duration won't have any impact on the system, prolonging the game won't have any effect.
  • The criteria are being measured for everybody, but how important they are is coming down to that particular situation. The system is measuring every game and looking at across the breadth of all the players what's the difference in the stats between a Bronze player and a Master player. There's more granularity in there, but it's looking to see what differentiates a lower skilled player and a higher skilled player and based on that it's weighing the 20 different factors in different ways.
  • The biggest factor is still winning or losing the game.
    • If you performed better, you're going to lose less points when your team loses a game.
    • If you, on theother hand, won the game, but your performance was not as good, you're going to gain less points.
  • Ult choice does not have any impact on MMR - it doesn't matter if you're playing a Varian and go full DPS or full tank. You're elevating some stats at the cost of others. The overall performance rating does not change a lot based on your ult choice, it's just about whether you're playing well overall.
  • If you're focusing on just 1 stat at the expenses of the other 19 stats, your overall performance will suffer. The system cannot be fooled to believe that you should be accelerated faster through the ranks when for example, you're trying to collect CC time as Kerrigan or Muradin.
  • Death time matters for all the stats.
  • Personal rank adjustment isn't going anywhere next Season, but they'd like to get rid of it in the future.
  • They'd like to make MMR visible in the future.
  • For the first period of time after a new Hero drops, performance-based matchmaking is shut off completely, until the machine learning system has gathered enough data to do that. The same applies to major Hero reworks or very specific scenarios (Cho'gall in Master League) - it uses the old system until there's enough data.
  • In case of any issues emerging with the new system, the engineers are able to shut the new matchmaking down entirely or for specific Heroes.
  • There may be loopholes that the developers will fix, but the new system effectively uses machine learning.
  • Players need to realize that by not doing anything (sitting at spawn) when a game is about to be lost will hurt them even more, because they will lose more points.
  • When handling feeders, the team is currently backed up by player reports and CS, but in the future, they'd like to have an automated system to evaluate feeding vs having a bad game.
  • Loss forgiveness is a new feature they'd like to roll out at some point to account for intentional feeders, leavers. The game either doesn't count as much for MMR or doesn't count at all for Ranked hits. 
  • The overall net effect is across the 20 different stats that are measured. If let's say you're in a double tank composition, your 1 stat (damage taken) may be lessened, because you are not the solo tank there, but your other stats will go up, because you've got more opportunities to do other things.
  • While inside vehicles (Dragon Knight, Garden Terror), your stats will go up really fast, but it's important to mention that the matchmaking compares your game to millions of other games on the same map and again, you sacrifice other stats in favor of the ones you gain in vehicles, so it's the same boucing act that balances itself out over time.
  • Revealing all the 20 criteria would cause poor game behavior, but the team works on a feature to show players areas they excelled in. No ETA for that.
  • Normal players don't need to be worry about the new matchmaking system. On the backend, it's making the matchmaker more accurate in the short period of time. It's okay just to worry about winning or losing. The overall game quality should increase, because players will be arriving faster at their MMR ranks.
  • The uncertainty boost in placement matches will not exist anymore in the new Season. Uncertainty is still there, but it's handled in a more dynamic way; it's based off of things like when you most recently played, streaks and so on.
  • Anybody who is nearby when a Mecenary camp is claimed will be granted credit toward the stat.
  • How they go about performance-based matchmaking in Heroes of the Storm and Overwatch is extremely different. 

Share this post


Link to post
Share on other sites

I have to say that I'm somewhat skeptical about how taking the dragon knight or garden terror wouldn't boost your in-game rating . Dragon Knight has CC, high damage AOE, and his main attack hits like a truck.

Share this post


Link to post
Share on other sites

I can only assume they will count your time in vehicle and compare it to people who had roughly same time in it.

I'm more interested about aspect that nubkeks said. At very high levels of play, you should be hiding in bush as a Warrior, effectively doing nothing, just making threat you are anywhere.

Share this post


Link to post
Share on other sites
12 hours ago, SleepySheepy said:

I'm more interested about aspect that nubkeks said. At very high levels of play, you should be hiding in bush as a Warrior, effectively doing nothing, just making threat you are anywhere.

Yeah, this and more. no matter what they do, there are a TON of in game situations where it's essentially impossible to measure how much impact a player is having. Even among "simple" stats like hero damage or damage absorbed, there can be a lot of difference in terms of the value of any given particular damage.

So, this is the trick: yes, all of these factors mean that performance based MMR will be less accurate than the current system (W/L only, over infinite matches, has perfect accuracy predicting how often you'll win or lose.) The trade-off, they hope, is speed. The problem with the current mmr system is that for it to represent player skill in any kind of accurate way you need hundreds and hundreds of games, which most players will just never get.

Overall, i'm optimistic. Machine learning has proved surprisingly effective at learning about these games in DOTA 2 (check out the recent series with OpenAI) so I think there's a good chance that this admittedly imperfect solution provides better results than the current "perfect but impractical" approach.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Stan
      Industrial District is a single-lane Overwatch-themed Battleground with shuffle pick and standard play. The first team to destroy the enemy Core wins. Complete 2 matches to earn a Loot Chest.
      Blizzard (Source)
      This week’s Heroes Brawl is Industrial District! Slide around and slug it out in the heart of Volskaya in this Overwatch-themed, single-lane Battleground featuring lots of conveyor belts and tons of action.
      Rules:
      Shuffle pick - Choose from one of three Heroes before entering the battle. Standard play - no talent or level restrictions. The first team to destroy the enemy Core wins! Rewards:
      Complete two matches of the Industrial District Brawl to earn a Loot Chest! Find out more about the Heroes Brawl game mode on our Heroes Brawl site; and as always, you can find more information on this week’s Brawl by clicking the Brawl Info button at the bottom of the play screen when preparing to queue for the Brawl game mode.
    • By Oxygen
      Our twentieth Heroes of the Storm Meta Tier list for the month of June is here!
      We present our twentieth Heroes of the Storm Meta Tier List for the month of June 2018.
      Welcome to Icy Veins's Meta Tier List for the June 12 patch. The goal of this list is to try and detail game's current metagame state. The prime goal of such lists is to inform players regarding popular and trending team composition drafting strategies (i.e. the drafting metagame). Although tier listings are generally the product of balance, many factors come into play when discussing the relative perceived strengths of heroes, including player regions, maps, play style, skill level, and, of course, personal perception. As such, any tier list—including this very one—should never be interpreted as gospel, but rather, as a guide to better grasp what to expect with regards to typical drafting experiences. One useful application of such lists is to allow you know which heroes to look out for in terms of practice and counterplay, ultimately improving your knowledge of the game.
      Using the list
       
      Current ranked mode map rotation
       
      Prime Tier
      Warrior Assassin Support Specialist Garrosh Fenix Malfurion  Zagara↑ Prime tier heroes are considered to be extremely strong in all situations, and show no obvious weakness. They are very often banned or picked right away, as they generally dictate the pace of most matches.
       
      Core Tier
      Warrior Assassin Support Specialist Arthas Chromie↑ Alexstrasza Abathur Blaze   Falstad Auriel↑ Azmodan Dehaka Genji↓ Deckard Nazeebo Diablo↓ Greymane Stukov↓   E.T.C.↓ Hanzo     Johanna Jaina     Muradin Junkrat     Stitches Li-Ming     Varian (Taunt) Lunara↑       Maiev↓       Malthael       Thrall↓       Tychus↑       Tracer     Core tier heroes are strong in a wide variety of situations and have few counter-picking possibility. They should form the core of your team, and be picked after Prime Tier heroes have been distributed.
       
      Viable Tier
      Warrior Assassin Support Specialist Anub'arak Alarak Brightwing Gazlowe Artanis Cassia Kharazim Medivh D.Va Gul'dan Li Li↓ Murky Leoric Kael'thas Lt. Morales↓ Sylvanas↓ Tyrael  Kel'Thuzad Lúcio↓ Xul Zarya Kerrigan Rehgar     Nova- Uther↓     Ragnaros       Samuro       The Butcher       Varian (Colossus Smash)↑       Varian (Twin Blades of Fury)↑       Valla       Zeratul+       Zul'jin     Viable tier heroes are generally well-rounded that have either fallen out of favour, or, inversely, are on the rise in popularity, due to the current Prime tier contenders.
       
      Niche Tier
      Warrior Assassin Support Specialist Chen Cho'gall Ana Probius Cho'gall Illidan Tassadar Sgt. Hammer Rexxar Raynor Tyrande↓     Valeera             Niche tier heroes have niche application on certain maps or for certain team compositions. They may also picked to "round out" your team composition when your team composition is missing out on key components, such as a "jungler" (mercenary camps), a solo laner, or solid waveclear.
       
      Lost Vikings Yrel Tier
      Warrior Assassin Support Specialist Yrel (new) - - The Lost Vikings The Lost Viking Yrel Tier heroes are deemed to be either considerably weaker than the majority of other Heroes, or much more challenging to play properly. Although they may situationally shine, these heroes are generally avoided by most players.
       
      Discussion
      I know I promised a new list for last week, but I generally try to give important changes about a week to set in somewhat before writing about them. Nexomania launching on a Friday (May 22) also threw my timing off slight as well, though I think it's safe to expect future events involving loot chests to fire off right before weekends for economic reasons.
       
      In other news, I brought up the idea of reworking how the list is presented a couple weeks ago. The first change I'd like to discuss pertains to role division. Since Blizzard is taking its sweetest time with reworking the class system, I'm looking for a way to better convey each hero's role. Concretely, this means I'm considering adding more categories and renaming some of the classic ones. This requires a good bit of consideration and some creativity, although I feel like the system I discussed in last month's tier list is solid enough to officially subject to everyone's scrutiny. Let me know how you feel about it, as I'm sure it can be improved.
       
      Tanks:
      Bruisers:
      Fighters:
      Stalkers:
      Marksmen: (and markswomen)
      Mages:
      Healers:
      Enablers:
       
      The second change I'm looking to make pertains to tiering. Although I've never been completely happy with it, some of the problems are becoming more and more apparent as the game is becoming more balanced, and as I spend time thinking about it. First, it fails to really make sense with regards to statistics. For instance, Genji, who stands at at a low 40% win rate, has been considered one of the strongest hero for over a year now. Why might that be? I don't know (well, I do...), but I certainly wish I could better convey this. Second, the names don't really make sense because they describe different states; niche and viable,  describe circumstances, whereas core and prime could be said describe a hierarchy. This doesn't really make sense as prime tier heroes are certainly viable, and as niche tier heroes can outperform so-called prime heroes under the right circumstances. Third, the system does not account for player skill level differences or regional preferences.

      I lack the data to sort heroes by league (bronze, silver, etc.), but even if I did have enough data, it would hardly tells the whole story (refer to the Genji anecdote above). The tiering system I'm considering goes like this, however:

      Metagame defining
      General pick
      Counter-pick
      Situational (map/team composition dependent)

      I'd probably add a Ban-worthy tag to certain heroes. For instance, I'd never ban Zagara over Chromie despite considering her generally stronger. Furthermore, I'd consider giving some heroes + and - tags to provide a certain sense of perceived power, though I'll have to consider carefully.

      As for the metagame discussion, it'll be coming shortly, as per usual. In the meantime, I'm looking forward to seeing how everyone feels about the changes.
    • By Stan
      It took a good while, but Heroes of Storm Highlights return with episode 110 of WTF Moments.
      In this video, we've got Alexstrasza miraculously surviving D.Va's Self-Destruct, Garrosh using Blaze's Bunker Drop to escape enemy forts after Stitches used Gorge on him, or Kel'Thuzad executing his Shadow Fissure at the right time to kill an escaping Falstad!
      Let us know your favorite moment in the comments down below and don't forget to submit your replays here for a chance to be featured in an upcoming episode!
      Previous Episodes
      WTF Moments Episode 109 WTF Moments Episode 108 WTF Moments Episode 106 & 107
    • By Stan
      The number of Battlegrounds available in non-Ranked play modes has been reduced until July 9, in order to maximize each player's exposure to Alterac Pass. The change does not affect Ranked Play, as Alterac Pass won't be available until the week of July 9 there.
      Blizzard (Source)
      Attention, recruits! We’ve received updated orders from the front in Alterac. The Horde and Alliance have finished setting up their forward operating bases in the Pass, so it’s time to take the battlefield by storm. We hope your training exercises with General Vanndar or General Drek’Thar have been successful, because Alterac Pass is now available for play in all non-ranked game modes, including Custom Games.
      NON-RANKED BATTLEGROUND POOLS TEMPORARILY REDUCED
      In order to maximize each soldier’s exposure to Alterac Pass, the number of Battlegrounds available for play in Versus A.I., Quick Match, and Unranked matchmaking queues have been temporarily reduced. This reduction will remain in place until the Echoes of Alterac event concludes on July 9. All of the Battlegrounds removed from the non-ranked pool will remain available for play in Custom Games.
      REDUCED BATTLEGROUND POOL Alterac Pass Braxis Holdout Cursed Hollow Dragon Shire Infernal Shrines Tomb of the Spider Queen Please note: Hero League and Team League Battleground pools have not been adjusted, as Alterac Pass will not be available in ranked play until the week of July 9.
      Grab your gear, set your loadouts, and report immediately to Alterac Pass for additional orders from your faction's General. We’ll see you on the front lines!
    • By Stan
      New Warcraft-themed skins are now available for purchase in Heroes of the Storm until July 9 and for the first time, new Hero skins included in the event bundles are also available in separate Skin packs.
      Blizzard (Source)
      Attention, recruits! We bring news from the front lines: Reinforcements have arrived for the Horde and the Alliance in the form of new Warcraft-themed bundles and skin packs. Read on to preview the items included in each, and then stock up your armory so you’re well prepared for the many battles that lie ahead.
      New Bundles
      Whether you’ve signed on with the Horde or the Alliance in the struggle for control over Alterac Pass, you can bolster your ranks with new skins, new mounts, and even a few Heroes by picking up one of our Warcraft-themed bundles. Each bundle is available for gem purchase until the Echoes of Alterac event concludes on July 9.
      For the Horde Bundle
      Heroes Skins Mounts Garrosh High Warlord Garrosh Raging War Wolf Rehgar Chaos Warlord Garrosh Raging Night Wolf   Frostwolf Warlord Garrosh Raging Timber Wolf   Drek'Thar Rehgar     Alterac Drek'Thar Rehgar     Draenor Drek'Thar Rehgar   For the Alliance Bundle
      Heroes Skins Mounts Muradin Vanndar Muradin Swift Stormpike Ram Varian Expedition Vanndar Muradin Swift Dark Iron Ram   Ironheart Vanndar Muradin Swift Stormwind Ram   Grand Marshal Varian     Kul Tiran Marshal Varian     Stromgarde Marshal Varian   Echoes of Alterac Bundle
      Heroes Skins Auriel Spirit Healer Auriel Johanna Ethereal Spirit Healer Auriel Junkrat Vengeful Spirit Healer Auriel   Warsong Johanna   Frostwolf Johanna   Mag'har Johanna   Sapper Junkrat   Darkspear Sapper Junkrat   Kezan Sapper Junkrat New Skin Packs
      Sometimes a lone soldier can make the difference between victory and defeat, especially when they’re armed with the right gear. Those who prefer individual personnel upgrades can pick up one of the many new skin packs that are available for gem purchase now until the Echoes of Alterac event concludes on July 9. Once these skin packs leave the collection, the shard costs to forge the skins they contained will be reduced.
      Spirit Healer Auriel Skin Pack
      Auriel Skins Spirit Healer Auriel Ethereal Spirit Healer Auriel Vengeful Spirit Healer Auriel High Warlord Garrosh Skin Pack
      Garrosh Skins High Warlord Garrosh Chaos Warlord Garrosh Frostwolf Warlord Garrosh Warsong Johanna Skin Pack
      Johanna Skins Warsong Johanna Frostwolf Johanna Mag'har Johanna Sapper Junkrat Skin Pack
      Junkrat Skins Sapper Junkrat Darkspear Sapper Junkrat Kezan Sapper Junkrat Vanndar Muradin Skin Pack
      Muradin Skins Vanndar Muradin Expedition Vanndar Muradin Ironheart Vanndar Muradin Drek'Thar Rehgar Skin Pack
      Rehgar Skins Drek'Thar Rehgar Alterac Drek'Thar Rehgar Draenor Drek'Thar Rehgar Nightslayer Valeera Skin Pack
      Valeera Skins Nightslayer Valeera 7th Legion Nightslayer Valeera Sunfury Nightslayer Valeera Grand Marshal Varian Skin Pack
      Varian Skins Grand Marshal Varian Kul Tiran Marshal Varian Stromgarde Marshal Varian Fight with honor and strength during your quest to bring victory to your faction in Alterac. We’ll see you on the front lines!