Stan

Performance-Based Matchmaking Explained (Interview)

Sign in to follow this  

4 posts in this topic

P94FCDT.jpg

Khaldor is back with yet another interview with Lead Designer Travis McGeathy about performance-based matchmaking. This time around, Travis answered a lot of interesting questions and responded to player concerns about the new system.

Previous Performance-Based Matchmaking Coverage

Khaldor's Interview

  • The goal of the new system is to accelerate you where you should be for your MMR. You won't get massively punished if you had one bad game.
  • The new performance-based matchmaking system constantly tracks 20 different stats (criteria) every game. They are used to tell how a particular player in a particular situation is doing. Everything is being sliced up for that situation. It's the Hero you're playing, the battleground you're on, the game mode you're in, the region and then there are modifiers for things like game time.
  • Game duration won't have any impact on the system, prolonging the game won't have any effect.
  • The criteria are being measured for everybody, but how important they are is coming down to that particular situation. The system is measuring every game and looking at across the breadth of all the players what's the difference in the stats between a Bronze player and a Master player. There's more granularity in there, but it's looking to see what differentiates a lower skilled player and a higher skilled player and based on that it's weighing the 20 different factors in different ways.
  • The biggest factor is still winning or losing the game.
    • If you performed better, you're going to lose less points when your team loses a game.
    • If you, on theother hand, won the game, but your performance was not as good, you're going to gain less points.
  • Ult choice does not have any impact on MMR - it doesn't matter if you're playing a Varian and go full DPS or full tank. You're elevating some stats at the cost of others. The overall performance rating does not change a lot based on your ult choice, it's just about whether you're playing well overall.
  • If you're focusing on just 1 stat at the expenses of the other 19 stats, your overall performance will suffer. The system cannot be fooled to believe that you should be accelerated faster through the ranks when for example, you're trying to collect CC time as Kerrigan or Muradin.
  • Death time matters for all the stats.
  • Personal rank adjustment isn't going anywhere next Season, but they'd like to get rid of it in the future.
  • They'd like to make MMR visible in the future.
  • For the first period of time after a new Hero drops, performance-based matchmaking is shut off completely, until the machine learning system has gathered enough data to do that. The same applies to major Hero reworks or very specific scenarios (Cho'gall in Master League) - it uses the old system until there's enough data.
  • In case of any issues emerging with the new system, the engineers are able to shut the new matchmaking down entirely or for specific Heroes.
  • There may be loopholes that the developers will fix, but the new system effectively uses machine learning.
  • Players need to realize that by not doing anything (sitting at spawn) when a game is about to be lost will hurt them even more, because they will lose more points.
  • When handling feeders, the team is currently backed up by player reports and CS, but in the future, they'd like to have an automated system to evaluate feeding vs having a bad game.
  • Loss forgiveness is a new feature they'd like to roll out at some point to account for intentional feeders, leavers. The game either doesn't count as much for MMR or doesn't count at all for Ranked hits. 
  • The overall net effect is across the 20 different stats that are measured. If let's say you're in a double tank composition, your 1 stat (damage taken) may be lessened, because you are not the solo tank there, but your other stats will go up, because you've got more opportunities to do other things.
  • While inside vehicles (Dragon Knight, Garden Terror), your stats will go up really fast, but it's important to mention that the matchmaking compares your game to millions of other games on the same map and again, you sacrifice other stats in favor of the ones you gain in vehicles, so it's the same boucing act that balances itself out over time.
  • Revealing all the 20 criteria would cause poor game behavior, but the team works on a feature to show players areas they excelled in. No ETA for that.
  • Normal players don't need to be worry about the new matchmaking system. On the backend, it's making the matchmaker more accurate in the short period of time. It's okay just to worry about winning or losing. The overall game quality should increase, because players will be arriving faster at their MMR ranks.
  • The uncertainty boost in placement matches will not exist anymore in the new Season. Uncertainty is still there, but it's handled in a more dynamic way; it's based off of things like when you most recently played, streaks and so on.
  • Anybody who is nearby when a Mecenary camp is claimed will be granted credit toward the stat.
  • How they go about performance-based matchmaking in Heroes of the Storm and Overwatch is extremely different. 

Share this post


Link to post
Share on other sites

I have to say that I'm somewhat skeptical about how taking the dragon knight or garden terror wouldn't boost your in-game rating . Dragon Knight has CC, high damage AOE, and his main attack hits like a truck.

Share this post


Link to post
Share on other sites

I can only assume they will count your time in vehicle and compare it to people who had roughly same time in it.

I'm more interested about aspect that nubkeks said. At very high levels of play, you should be hiding in bush as a Warrior, effectively doing nothing, just making threat you are anywhere.

Share this post


Link to post
Share on other sites
12 hours ago, SleepySheepy said:

I'm more interested about aspect that nubkeks said. At very high levels of play, you should be hiding in bush as a Warrior, effectively doing nothing, just making threat you are anywhere.

Yeah, this and more. no matter what they do, there are a TON of in game situations where it's essentially impossible to measure how much impact a player is having. Even among "simple" stats like hero damage or damage absorbed, there can be a lot of difference in terms of the value of any given particular damage.

So, this is the trick: yes, all of these factors mean that performance based MMR will be less accurate than the current system (W/L only, over infinite matches, has perfect accuracy predicting how often you'll win or lose.) The trade-off, they hope, is speed. The problem with the current mmr system is that for it to represent player skill in any kind of accurate way you need hundreds and hundreds of games, which most players will just never get.

Overall, i'm optimistic. Machine learning has proved surprisingly effective at learning about these games in DOTA 2 (check out the recent series with OpenAI) so I think there's a good chance that this admittedly imperfect solution provides better results than the current "perfect but impractical" approach.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Stan

      Sylvanas, Rehgar and Uther are on sale next week. Check out the full list of featured items added in next week's Collection Update!
      Blizzard (Source)
      Weekly Collection Update: Feb 27 – Mar 6, 2018
      Our next Collection Update will begin on Tuesday, February 27! Check out the list of featured items and Hero discounts below.  Upcoming Hero Sales
      Sylvanas — Sale Price: 312 Gems Rehgar — Sale Price: 312 Gems Uther — Sale Price: 250 Gems  The following Skins and Mounts will enter the Featured Item Rotation, and will be unlockable using Gems until next week's Collection Update: Upcoming Featured Skins
      Ash Primal Queen Kerrigan  Crimson Storm Wolf Rehgar  Night Explorer Li Li  Hyper Turbo Tracer  AzGul’dan  Viridian Super Sonya  Spectre Thunder Guard Zarya  Wild Raven Sylvanas  Cobalt Cyb’arak Anub’arak  Blazing Lil’ Ragnaros  Upcoming Featured Mounts
      Tan Horse  Cloud Serpent  Hunter Space Lord’s Starbreaker
    • By Stan

      10 Sgt. Hammers on Towers of Doom. Thrusters are always active and all players start at level 10 with no talent restrictions. Play two matches of Hammer Time to collect a Lunar Loot Chest.
      Blizzard (Source)
      This week’s brawl is Hammer Time! An explosive twist on a typical match, Hammer Time lets you call down the thunder and claim Belltowers on Towers of Doom with 10 Sgt. Hammers.

      Rules
      Queue up and battle it out on Towers of Doom with either Sgt. Doomhammer or World War Sgt. Hammer. Core health for both teams halved to 20 Modified Standard play No talent restrictions, but all players start at level 10 Extended beginning game time by 15 seconds to allow for more time to choose talents Thrusters are always active for all players! Unsieging from Siege Mode is now instant for all players! The first team to claim enough Belltowers to destroy the enemy Core is victorious! Rewards
      Complete two matches of Hammer Time to earn a Loot Chest!
      Find out more about the Heroes Brawl game mode on our Brawl site; and as always, you can find more information on this week’s Brawl by clicking the Brawl Info button at the bottom of the play screen when preparing to queue for this exciting new game mode.
    • By Stan

      Voice chat is finally available in Heroes of the Storm and this post by Blizzard will guide you through the basics of the system.
      All players must opt-in to join the team voice channel at the start of a game or upon entering a draft lobby. If you don't have a microphone, you will still be able to hear your team. The voice chat tab in the options menu can be used to adjust all settings to your liking. Use the Test button to hear how you'll sound in-game and don't forget to set up your Push-to-Talk Key.
      Blizzard (Source)
      We’re excited to announce that voice communication is now available in Heroes of the Storm! Whether you find yourself in a draft lobby, partied with a friend or two, or with your whole squad in Team League, voice communication will free your hands from the shackles of typing and make it easier than ever to coordinate with your teammates. Before you head in-game to try it out, take a moment to learn how voice channels work in Heroes and get to know the various settings you can adjust to make voice communication your own.
      Team and Party Voice
      We’ve added voice channels to Heroes of the Storm to accommodate both party voice and team voice. If you’re in a party, you’ll automatically be added to your party’s voice channel, and only party members will hear what you have to say. The opposite is also true: if you’re in a partial party and using the party voice channel, you’ll be unable to speak to or hear allies outside your party unless you switch to the team voice channel.
      By default, all players must opt-in to join the team voice channel at the start of a game or upon entering a draft lobby. A notification will appear in chat to remind you that team voice is available, and you can click it to easily join the channel. Keep in mind that voice channels—even party voice—are completely optional. You can use the new voice chat tab in the options menu to adjust auto-join settings and notification sounds to your heart’s content.
      Voice Settings
      Digging a bit further into voice settings, you can click the dropdown menus at the top to pick which device you’ll use to hear your allies’ voices, and set up the microphone you’d like to use for your own. You can click the Test button to hear how you’ll sound to your teammates in-game. However, even if you don’t have a microphone, you can still join party and team voice channels to listen-in as your allies make plans and shotcall.
      As mentioned previously, the checkboxes under Auto-Join Voice and Notification Sounds on the left side of the voice chat options menu can be used to select which voice channels you’d like to join automatically during each game, as well as to enable or disable any voice notification sounds.
      Click the Transmission Mode dropdown to decide whether you’d like to set your microphone to Push-to-Talk or Open Mic. When using push-to-talk, your allies will only hear you while you’re holding down your push-to-talk hotkey (the ` key by default). With open mic, they’ll hear everything you have to say. If you do opt for an open mic, you can adjust the sensitivity slider to stop your microphone from transmitting sound under a certain volume. Finally, run a quick test to make sure your mic is sounding good and your sensitivity is set correctly—broadcasting noise is a surefire way to get muted by your teammates!
      In addition to push-to-talk, you can set up hotkeys to quickly switch between party and team voice channels, or to turn your microphone on and off. 
      In-Game Voice
      Outside of the voice chat options menu, you’ll see a voice widget appear in the top-left corner of most game menus. Clicking this widget will allow you to quickly join, switch, or leave a voice channel. Once you enter a game, you’ll find this widget on the tab screen.
      You’ll also find microphone and speaker icons across from player names on the tab screen, which you can click to mute or unmute allies individually.
      We’ve also added a new reporting option called Abusive Voice that you can use to report players specifically for misbehavior over voice. Additionally, we’ve implemented a voice silence penalty that’s separate from chat silence penalties. This means that players who are frequently reported for Abusive Voice can receive a voice silence, which will prevent them from speaking in team voice for the duration of that penalty. They can, however, still join the team voice channel to listen to allied comms, and can still join and use the party voice channel normally.
      We hope the addition of voice in Heroes of the Storm will help you and your allies communicate more effectively on the road to your next string of victories, and perhaps make a new friend or two along the way.
      Until next time, we’ll see you in the Nexus!
    • By Stan

      Blizzard implemented voice chat and applied balanced changes and various bug fixes in this week's patch.
      Voice chat is now enabled on live servers and in addition to bug fixes, the following Heroes received balance changes:
      Maiev: The latest changes are aimed at lowering her survivability and burst damage throughput. Malthael: Healing from Soul Ripping Minions decreased, Hero-only healing increased. Mana cost of Soul Rip has been increased to 25. Zeratul: Changes to basic & heroic abilities are meant to redistribute his damage, while giving the Hero a slight buff. Xul: Base stats have been buffed and split-push potential has been toned down. Sonya: Balance changes for Sonya were supposed to go live on January 24, but Blizzard's implemented them just now. The changes include a basic attack damage reduction, Poisoned Spear & Nerves of Steel talent adjustments. Blizzard (Source)
      We’ve just released a patch to the live servers in order to implement Voice Communication, as well as apply some balance updates and bug fixes.
      General
      Voice Communication
      Voice Communication has been added to Heroes of the Storm! Read the Voice Communication blog for more information about how to use it in-game. Heroes
      Assassin
      Maiev

      Stats
      Health reduced from 2236 to 2150 Health Regeneration reduced from 4.66 to 4.48 Abilities
      Vault of the Wardens (Trait) Cooldown increased from 8 to 9 seconds Fan of Knives (Q) Damage reduced from 170 to 162 Talents
      Level 16 Armored Assault (Passive) Damage bonus reduced from 25% to 20% Vengeful Knives (E) Bonus damage per Hero hit increased from 1% to 1.2% Malthael

      Abilities
      Soul Rip (Q) Mana cost increased from 20 to 25 Healing reduced from 36 to 25 per target hit Healing from Heroic targets increased from 3% to 4% of the Hero’s maximum Health Tormented Souls (R) Cooldown reduced from 100 to 80 seconds Talents
      Level 4 Die Alone (Q) Damage bonus reduced from 75% to 50% Level 16 Soul Collector (Q) Bonus range reduced from 50% to 25% Thrall

      Abilities
      Chain Lightning (Q) Bounce range increased by 20% Zeratul

      Abilities
      Cleave (Q) Damage increased from 200 to 212 Singularity Spike (W) Damage increased from 228 to 240 Void Prison (R) Mana cost reduced from 100 to 80 Might Of The Nerazim (R) Basic Attack damage bonus reduced from 40% to 30% Specialist
      Xul

      Stats
      Health increased from 1900 to 2000 Health Regeneration increased 3.957 to 4.168 Basic Attack damage increased from 104 to 110 Abilities
      Raise Skeleton (Trait) Skeletal Warrior Health reduced from 240 to 225 Skeletal Warrior damage reduced from 23 to 21 Bone Prison (E) Cooldown reduced from 12 to 10 seconds Mana cost reduced from 80 to 70 Talents
      Level 7 Trag'Oul's Essence (Trait) Mana restoration per Skeletal Warrior attack reduced from 0.5% to 0.2% of Xul’s maximum Mana Support
      Tyrande

      Abilities
      Light of Elune (Q) Cooldown reduced from 12 to 11 seconds Warrior

      Sonya

      Stats
      Basic Attack damage reduced from 88 to 84 Talents
      Level 7 Poisoned Spear (Q) Damage reduced from 125% to 100% of Ancient Spear’s damage. Level 16 Nerves of Steel (Active) Duration reduced from 5 to 3 seconds Shield amount reduced from 30% to 25% of Maximum Health Cooldown reduced from 70 to 60 seconds Bug Fixes
      Heroes, Abilities, and Talents
      Diablo: Lightning Breath sound effects will no longer stop playing before the Ability is finished channeling. Junkrat: Heroes killed by Concussion Mine will no longer respawn at their death location and then slide across the Battleground. Junkrat: Issuing movement commands during Rocket Ride will no longer cause Junkrat to become separated from his Mount. Junkrat: Movement commands that the player did not issue will no longer appear on the terrain in front of RIP-Tire’s model. Junkrat: Targeting unpathable terrain with Rocket Ride will no longer cause Total Mayhem Grenades to visually persist on the Battleground. Junkrat: Concussion Mine can no longer displace Leoric from Wraith Walk’s starting location if the Mine is detonated in the same moment that Wraith Walk ends. Li Li: After learning the Lighting Serpent Talent, Cloud Serpent attacks that are bouncing to additional targets when the Ability expires will now correctly deal damage. Lt. Morales: The tooltip for the Healing Beam icon next to Lt. Morales’ in-game portrait will now correctly display the Ability’s cooldown duration. Malfurion: An ally who is affected by multiple Regrowths will now be correctly healed by Moonfire and have Stuns, Roots, and Slows removed by Nature's Cure. Movement Abilities: Certain movement-based Abilities, like Greymane’s Darkflight or Chen’s Flying Kick, will no longer fail to move the Hero toward a target currently affected by Varian’s Taunt or Garrosh’s Warlord's Challenge. Sgt. Hammer: Can now correctly deploy Biotic Emitters on Volskaya Foundry while in Siege Mode. The Butcher: Fixed an issue that prevented The Butcher from collecting Meat while piloting the Triglav Protector, Dragon Knight, or Garden Terror.