Stan

Comments on Performance-Based Matchmaking by Lead Systems Designer Travis McGeathy

11 posts in this topic

MSKG7EUORNAF1456364705258.jpg

Redditor Elitesparkle gathered insightful comments by Lead Systems Designer Travis McGeathy about the upcoming performance-based matchmaking system.

Heroes of the Storm will be soon updated with a new MMR system that takes individual gameplay into account. The current system is limited to wining / losing a game. The new system is more accurate, because it uses various parameters to calculate your MMR. Lead Systems Designer Travis McGeathy answered some interesting questions about the new system over on reddit.

Screenshots from the What's Next Panel

The 2018 update is scheduled for mid-December.

CPcfp1u.jpgUTdXHOA.jpgsN0oZWC.jpg

Blizzard LogoBlizzard (Source)

Any changes also planned for uncertainty?

Uncertainty doesn't exist in the same form in the new system. The core concept of something that limits how much MMR can adjust on a per-game basis is there, but the factors that go into it are different.

Looks like new system promotes maining a few heroes rather than flexibility, true or not, and how big is the impact if true?

The most important thing is still winning the game. That hasn't changed. Sometimes, that means sticking to heroes you know really well. Other times, that means having flexibility in your hero roster and playing the heroes that best fit the situation.

Might be a stupid question, but if it's only about winning, why do we have stats-based MMR? why the win isn't the only factor? Isn't the concept of stats-based MMR less accurate than the winrate itself to calculate your odd of winning?

It's about speed more than accuracy. The current system, based entirely on wins vs loses, works, but because your skill is diluted in a team of 5, unless you are dramatically better (or worse) than the rest of the team, it can take a large number of games before the difference in skill translates to more wins (or loses) and moves your MMR to where it should be. By factoring in individual skill to the equation, it can arrive at the proper MMR much faster which means the matchmaker can make better matches overall.

Is draft/banning taken into account? Choosing the right hero in draft may be more important than performing better than peers on a certain hero.

Drafting matters in the same way as the above statement. Drafting is a skill and how well you draft affects your win rate.

All in all, do you have some additional insight into what parameters are used? How zoning, peeling, bodyblocking, skillshot blocking is taken into account? How will Force Wall-Tassadar be able to compete with Archon-Tassadar?

We have great insight into what parameters are used...but aren't going into that level of detail externally.

Also do you plan to add more parameters in the mix?

We'll add more parameters as it makes sense.

There's some concern about how reliable is data on low played heroes — maybe like Chen, Rexxar, TLV, anything on that? (Though I can't imagine any answer here but "we think our data is reliable enough" :D)

We have more than enough data on all existing heroes. For new heroes or major reworks, the performance-based adjustments will basically be disabled until enough data exists.

Are we right to assume that new MMR calculation would be applied to all game modes? For QM, it looks like a step in the direction of per-hero MMR, do you think we may end up there, or at least do you think it may be viable to maintain?

It will apply to all game modes that have MMR.

Can you win a game yet still lose MMR points? IE: a player goes full tilt, intentionally tries to throw. But the other 4 players are just dominating and still win the game.

No. You'll only gain points on a win and lose them on a loss. If you get full carried for a game, you won't lose points, but you also aren't going to gain very many either.

I think people's main concerns are along the lines of "who determines which stats are important" and "why these stats and not these other ones". I assume you guys are running some kind of adaptive machine learning, but most people are not going to understand that so maybe some kind of brief summary into how that works can help to alleviate the main concerns. Just a suggestion.

Quite right. For the "who determines which stats are important", its the players. We've chosen the stats to monitor, but which ones are important for that situation is something we are measuring, not determining ourselves.

About revealing MMR and the general clarity. Assuming "no" on "if you're going to reveal MMR" (:D) I'd like to ask whether we would be able to see when the adjustment takes place on the post-match screen, like we see PRA now.

We won't have visible MMR for this update, but I do still want to do it in the future. For now, you'll mainly see the effects through rank points. We're planning to swap out personal rank adjustment for a more direct performance adjustment which will give better insight while filling a similar role of keeping rank close to MMR. I'm not sure right now whether that will be for this season or the next yet.

Spot on and part of why visible MMR hasn't been a big priority for us but, there are some advantages to it:
- Currently, rank is the only visible indicator of skill. This forces us to link rank to MMR so they don't diverge too much because players get upset when they see people of different rank in their games even if they are actually the same skill. That's not ideal and leads to oddities with matchmaking when rank and MMR do diverge.
- Even though we don't show MMR, players have created other sources to approximate their MMR. The information isn't very accurate, but being the only source of knowledge, it gets cited frequently when players feel there are issues with matchmaking. There's two issues here: first, it gives a false impression that matchmaking is poor for some players and second, it makes it harder to understand when there are actually issues with matchmaking since the real issues can get buried. Effectively, for players that really care about MMR, we already have most of the downsides associated with visible MMR without having the upsides of it being accurate information.

But what order of magnitude are you all thinking? 10 points? 20, 50? Just trying to get a rough understanding of the overall impact it might have on gaining.

That's a tuning point we haven't settled on yet.

The system, unless specifically programmed to do otherwise, will think a death at the last moment is extremely meaningful since it is normally a 60s death timer.

Time is factored into the measurements. If you're dead for the last few seconds of a match, there wouldn't be any measurable impact relative to everything else that happened that game.

Regarding the new performance based MMR changes, does this mean if my team has captured the winning altar on Towers of Doom and for the fun of it I walk into the enemy death zone and die, my MMR will be adversely affected from this little bit of fun? Or perhaps there are 5 enemies killing my core and there are 4 allies dead. I have no chance to defend the core but rather than try and risk receiving an extra death I won't lose as much MMR if I stay in base? Curious about this.

Everything is by time, including deaths. If you die in the last second of a match, its not going to mean much. Even a full death in the last minutes would really only be a minor modifier overall once diluted against your overall performance in the game. The biggest factor for MMR adjustment is still whether you won or lost. Your overall performance is secondary to that and any particular moment is just a small part of that overall performance.

Does this mean that winning a game quickly and playing well will reward the same personal adjustment as having good stats in a long game? Some people are concerned that players will want to artificially extend game lengths to try and pad their stats before winning. Especially in games where the opponent is greatly outmatched.

Yeah, definitely. Everything factors game time into it for that exact reason.

Share this post


Link to post
Share on other sites

My conclusions:

-Win ratio is the most important

-Game lenght does not affect performance

-It might promote the usage of the same hero(he tried not to give a clear answear so this means the answear wouldn't please the players)

-We still don't know how MMR is calculated.

Share this post


Link to post
Share on other sites
11 hours ago, Stan said:

The most important thing is still winning the game. That hasn't changed. Sometimes, that means sticking to heroes you know really well. Other times, that means having flexibility in your hero roster and playing the heroes that best fit the situation.

That will force people to actually learn the Heroes in Try Mode or VS AI before compromising entire team compositions because they don't know how things work. For instance, we all know that "Nova main" who doesn't even know that the red eye icon means you've been detected.

Share this post


Link to post
Share on other sites

I was hoping new system would go away from winning being only major parameter, because it's team statistic, not something that is entirely upon you.

I'll only mention one other system from I game I used to play, WN8 for World of Tanks.
This measure is community driven, nothing official. Despite being system, well, that calls out for  stat-whoring people (don't know other word), it's really hard to do so that you aren't actively winning. Main component to WN8 was doing damage and assisting others in doing damage, only way to break system was to be able to somehow do a lot of damage on game that will be lost, really hard to do. 
Winrate is part of it, but it only made a small portion.

I'm not sure why are they persistent on winning being main driver of what happens with your MMR.
 

Can you win a game yet still lose MMR points? IE: a player goes full tilt, intentionally tries to throw. But the other 4 players are just dominating and still win the game.

No. You'll only gain points on a win and lose them on a loss. If you get full carried for a game, you won't lose points, but you also aren't going to gain very many either."

They stated it themselves, it's still not accurate in those extremities.

Edited by SleepySheepy

Share this post


Link to post
Share on other sites
2 hours ago, SleepySheepy said:

I was hoping new system would go away from winning being only major parameter, because it's team statistic, not something that is entirely upon you.

I'll only mention one other system from I game I used to play, WN8 for World of Tanks.
This measure is community driven, nothing official. Despite being system, well, that calls out for  stat-whoring people (don't know other word), it's really hard to do so that you aren't actively winning. Main component to WN8 was doing damage and assisting others in doing damage, only way to break system was to be able to somehow do a lot of damage on game that will be lost, really hard to do. 
Winrate is part of it, but it only made a small portion.

I'm not sure why are they persistent on winning being main driver of what happens with your MMR.

The main difference is that World of Tanks has 15v15 matches, where teamwork is not as crucial as in HotS (that's not to say that it is not important in WoT), and since there is no draft phase, you don't have to be able to adapt to enemy team's composition. You can simply roll with a Tier X SPG and get a high rating. If people were to do that in HotS, the team compositions would be pretty bad, so, as they said, those are reflected in the win rate. 

Share this post


Link to post
Share on other sites
4 hours ago, SleepySheepy said:

I was hoping new system would go away from winning being only major parameter...
 

Can you win a game yet still lose MMR points? IE: a player goes full tilt, intentionally tries to throw. But the other 4 players are just dominating and still win the game.

No. You'll only gain points on a win and lose them on a loss. If you get full carried for a game, you won't lose points, but you also aren't going to gain very many either."

They stated it themselves, it's still not accurate in those extremities.

And they did. Now it's not the only major parameter, but one of many. But it's still a priority, which is very logical, isn't it? Or you were hoping that even when losing a match you could still EARN mmr? I'm sorry but I couldn't help reading that between your lines.

And just skipped the rest of your post about WoT. It's completely irrelevant simply because comparing games of the same genre is a risk of running into wrong argumentation, let alone mentioning games of completely different nature.

Share this post


Link to post
Share on other sites

I understand winning and losing being the primary parameters.  If they weren't, EVERYONE would just prioritize Nova and try and go 15-0 every match, losses be damned.

Speaking as someone who hovers around a 50% win rate (keeping in mind I play my lowest ranked hero to fill quests), I have a lot of games with people who, frankly, have no idea about even basic strategy.  I also go on a ton of long (10 games+) loosing streaks.  And I almost never carry.  HotS has been VERY frustrating at times as a result.  As far as I'm concerned, MMR matchmaking can't get much worse then it currently is.

Share this post


Link to post
Share on other sites
15 hours ago, gamerk2 said:

I understand winning and losing being the primary parameters.  If they weren't, EVERYONE would just prioritize Nova and try and go 15-0 every match, losses be damned.

Speaking as someone who hovers around a 50% win rate (keeping in mind I play my lowest ranked hero to fill quests), I have a lot of games with people who, frankly, have no idea about even basic strategy.  I also go on a ton of long (10 games+) loosing streaks.  And I almost never carry.  HotS has been VERY frustrating at times as a result.  As far as I'm concerned, MMR matchmaking can't get much worse then it currently is.

Yep, matchmaking in HotS is terrible, that's why I quit. Even though I pretty much always had the best stats (xp, damage, ka/d ratio, heal) in lost matches, this system is not making me coming back.

Enough is enough.

Edited by Esmer

Share this post


Link to post
Share on other sites
Quote

 I assume you guys are running some kind of adaptive machine learning, but most people are not going to understand that so maybe some kind of brief summary into how that works can help to alleviate the main concerns. Just a suggestion.

Why people are so afraid to ask the damn question ? That question is: when will you reveal what is taken into consideration when calculating one's MMR ?

If Blizzard want HotS to be a competetive game then how players are supposed to be competetive when they don;t know how system works ?? This irritates me so much :/ If the system is as good as they describe it then there should be no problem with telling people how does it work exactly. If they don't want to reveal this info then there is sth wrong with it imo.

Share this post


Link to post
Share on other sites

I hope HotS dies out as a result of the poor matchmaking that they've had since the start. Came over to see the pretty models left after hitting master q'ing as Aba/Murky. This game is the pretty version of a League of Legends shortbus.

Share this post


Link to post
Share on other sites
On 14. 11. 2017. at 5:21 PM, Jonar said:

And they did. Now it's not the only major parameter, but one of many. But it's still a priority, which is very logical, isn't it? Or you were hoping that even when losing a match you could still EARN mmr? I'm sorry but I couldn't help reading that between your lines.

And just skipped the rest of your post about WoT. It's completely irrelevant simply because comparing games of the same genre is a risk of running into wrong argumentation, let alone mentioning games of completely different nature.

Yeah, you "read" it right. Seems silly to me that MMR will be positive or negative just depending on win/loss.

Is that wrong thinking? I would say it's not, but of course, I could be wrong.

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Stan
      It's Mal'Ganis week! The Hero is on rotation for the first time, so remember to level him to level 5 to earn gold if you haven't done that already.
      This week's Heroic Deals include Tyrael and Sgt. Hammer. Don't forget about special promotions (360-Day Boost & Hero Bundles) that are available until November 27. 
      Blizzard (Source)
      Free-to-Play Hero Rotation: November 20, 2018
      Raynor Muradin Sonya Nazeebo Uther Jaina Stukov Arthas Auriel Kerrigan Probius (Slot unlocked at Player Level 5) Tracer (Slot unlocked at Player Level 10) Valeera (Slot unlocked at Player Level 15) Mal'Ganis (Slot unlocked at Player Level 20)
    • By Stan
      Tyrael and Sgt. Hammer will be on sale next week along with the 360-Day Boost through November 27!
      If you haven't bought the Boost yet you still have one week to do so. Mounts available next week include Big Top Ringmaster’s Pride and Platinum Tyrael’s Charger.
      Blizzard (Source)
      Every Tuesday, we place a number of Heroes on sale, and swap a fresh set of cosmetics into the featured item rotation. Check out this week’s items and then head in-game to pick up anything that catches your eye.
      Heroic Deals: November 20 – November 27, 2018
      Our next set of Heroic Deals started on Tuesday, November 13! Check out the list of upcoming featured items and Hero discounts below.
      Upcoming Hero Sales
      Tyrael— Sale Price: 250 gems Sgt. Hammer — Sale Price: 312 Gems The following skins and mounts will be added to the featured item rotation and will become purchasable with gems until this round of Heroic Deals comes to an end:
      Upcoming Featured Skins
      Demonic Tyrael War World Sgt. Hammer Balespider Gul’dan Violet Flying Monkey Brightwing Twilight Archangel Diablo Ghost Illidan Upcoming Featured Mounts
      Big Top Ringmaster’s Pride Platinum Tyrael’s Charger
    • By Stan
      The first ever PvP brawl returns to the Nexus. It's time to Escape from Braxis!
      A preset roster will be available for this brawl. If a Hero dies, they can be resurrected by Allies, but when all players die, the mission is failed. Everybody starts at level 10 and has access to their full talents with a few exceptions. Complete 3 matches of Escape from Braxis to receive a Loot Chest. Blizzard (Source)
      Things aren’t looking great, soldier. You’re stranded on Braxis and the UED is overrun. The Omegaswarm has surrounded you on all sides, and your only shot at survival is to press through the onslaught, make it to the extraction point, and get off this forsaken rock. Will you fall victim to the Swarm, or can you work together and Escape from Braxis?
      Rules:
      At the beginning of the Brawl, your team will have the following roster of Heroes to choose from: Warrior: Artanis, Arthas, Johanna, Muradin, Sonya, Tyrael, Varian, Zarya Support: Brightwing, Kharazim, Li Li, Lt. Morales, Lucio, Malfurion, Rehgar, Uther Specialist: Azmodan, Gazlowe, Sgt. Hammer, Nazeebo, Xul Assassin: Cassia, Falstad, Greymane, Gul'dan, Illidan, Jaina, Kael'thas, Kel'Thuzad, Ragnaros, Raynor, Thrall, Valla Player Death: When a Hero dies, they can be resurrected by Allies after a short channel time. If all players die, the mission is failed. Players start at Level 10 and have access to their full talents with a few exceptions (i.e. Bribe) Enemy zerg Heroes will spawn periodically throughout the brawl and will drop new active items exclusive to Escape from Braxis. Push through the onslaught and survive to Escape from Braxis and win the Brawl! Rewards
      Complete three matches of Escape from Braxis to earn a Loot Chest!
    • By Stan
      The latest patch is now live with Orphea, visual updates to Dragon Shire, and new Hero bundles!
      Highlights
      Orphea received additional tweaks today in the live version of the patch. The new Hero is available to all BlizzCon attendees and Virtual Ticket holders free of charge. Don't forget to check out our Orphea build guide (will be up soon). Alextrasza’s Life-Binder (R) and Rehgar’s Ancestral Healing (R) abilities are no longer cancelled under any conditions other than enemy abilities that remove the target from play. Stimpacks are now called Boosts and the 360 Day Boost is available at a 33% reduced price for the next two weeks along with new Hero Bundles. Blizzard released a new digital comic about Orphea yesterday. Blizzard (Source)
      Our newest patch, featuring Orphea, is now live! Read on for a detailed breakdown of what’s new in the Nexus. 
      NOTE: Orange text indicates a change between the PTR and Live notes.
      General
      Boost What was once called a Stimpack is now a Boost, and in addition to buffing your own experience and Gold gains at the end of a match, all players will receive 5% bump in experience for each player in their match who had an active Boost. That mean players can receive up to a 45% XP bonus if all 10 players in game have a Boost active! In addition to stacking Boost XP buffs, you’ll now be able to retroactively Boost your Gold and XP gains at the end of a match by applying a Boost from the Score Screen. Design
      Channeled Healing Abilities Alextrasza’s Life-Binder (R) and Rehgar’s Ancestral Healing (R) abilities are no longer cancelled under any conditions other than enemy abilities that remove the target from play (Devouring Maw, Gorge, Cocoon.) Spell Power Modifiers The following abilities and talents have been updated to have any percentage-based spell power modifiers stack additively when calculating damage: (This is the default behavior for all other abilities and talents unless otherwise stated in the tooltip) Zul’jin’s Buzzsaw (R) talent Hanzo’s Dragonstrike (R) center damage effect Gul’dan’s Rampant Hellfire (Q) talent The following abilities have had their tooltips updated to reflect that percentage-based spell power modifiers will stack multiplicatively when calculating damage: Kael’thas’ Phoenix (R) ability Nazeebo’s Vile Infection (Passive) talent Li Li’s Lightning Serpent (W) talent Li-Ming’s Arcane Orbit (W) talent Battlegrounds
      Experience gains from Mercenary Camp captures have been unified. All Mercenary Camps now grant all Experience on capture. Dragon Shire: Dragon Shire has received updated Battleground art (including an updated Dragon Knight) New Hero: Orphea (Assassin)
      Rejecting the dark power her father had begun to consort with, Orphea tried to run away from the Raven Lord's influence. But after witnessing King's Crest fall to his machinations, she was convinced she needed to stand and fight. She now vows to protect the innocent – as a Hero of the Nexus.
      Trait
      Overflowing Chaos (Trait) Hitting an enemy Hero with a Basic Ability grants 1 Chaos. Chaos can stack up to 3 times. While Orphea has Chaos, her Basic Attacks against Heroes consumes all Chaos, dealing 50% increased damage per stack, and healing for 100% of the damage dealt. Basic Abilities
      Shadow Waltz (Q) After 0.5 seconds, deal 150 damage to enemies in a line. Hitting a Hero with Shadow Waltz sets its cooldown to 2 seconds and causes Orphea to dash a short distance upon moving. Chomp (W) After 0.6 seconds, deal 285 damage to nearby enemies in front of Orphea. Dread (E) Release a wave of dread that deals 85 damage to enemies hit. Dread erupts 0.75 seconds after reaching the end of its path, dealing 175 damage and Slowing enemies in the area by 25% for 2 seconds. Heroic Abilities
      Eternal Feast (R) After 1.5 seconds, deal 210 damage in an area. Eternal Feast repeats every 1 second as long as it hits an enemy Hero. Crushing Jaws (R) After 1.25 seconds, pulls enemies in an area towards the center, dealing 275 damage and Stunning them for 0.75 seconds. User Interface
      Additional space for Heroes has been added to the Hero select screen. Battlegrounds with mechanics that add additional (default hotkey: F) buttons (Warhead Junction, Snow Brawl, etc.) have had the appearances of their panels unified. A new “Bundles” page has been added to the Collection tab. Heroes
      Assassin

      Gul'dan
      Talents Level 16 Rampant Hellfire (Q) Damage bonus increased from 8 to 10%
      Orphea
      Stats
      Maximum Base Health reduced from 1750 to 1662 Health Regen reduced from 4.09 to 3.46 Abilities
      Shadow Waltz (Q) Base cooldown increased from 5 to 6 seconds Mana cost increased from 25 to 40 Mana refund on hit increased from 25 to 40 Talents
      Level 7 Ravenous Hunger (W) Damage per Minion kill reduced from 3 to 2 Damage per Hero kill reduced from 60 to 40 Level 20 Eldritch Conduit (Trait) Spell Power per stack reduced from 5 to 3% Spell power max stack bonus reduced from 50 to 30%
      Collection
      NOTE: Many of the new items announced at BlizzCon 2018 will not be immediately available with this patch but will be added to the game in the coming weeks. New Bundles
      The following new bundles are only available for a limited time! Orphea Heroic Bundle The following bundles have been added to the collection: Welcome to the Nexus Bundle Heroic Reinforcement Bundle New Announcers
      Orphea Ka Neithis New Mount
      Raven Runner Haunted Raven Runner Bloodmoon Raven Runner New Skins
      Orphea Hallowed Orphea Verdant Orphea Slacker Orphea RCHS Slacker Orphea Spoiled Slacker Orphea New Portraits, Sprays, and Emojis
      Several new emoji packs, sprays, and portraits have also been added to the game.  Bug Fixes
      General
      Fixed an issue that caused some Shielding and Healing values to track improperly on the score/stats screen. Fixed an issue allowing some summoned units to grant vision of Heroes inside brush if placed at the edge of the brush. Fixed an issue causing new item counts to appear over the wrong category. Art
      Fixed an issue causing some spell effects associated with taunt animations to appear after the animation was canceled. Whitemane: Fixed an issue causing Witch Whitemane’s custom mount animation to cease playing after she died once in a match. User Interface
      Fixed a number of tooltip issues and inaccuracies across the game. Fixed an issue with the Boost button being one pixel lower than the adjacent end-of-match buttons. We gathered the team, pooled resources, and after a lot of squinting at our monitors and a brief search for a magnifying glass, we’ve located the issue and boosted up the boost button to be in line with its neighbors. Thanks for paying such close attention! Heroes, Abilities, and Talents
      Kerrigan: Fixed an issue that disabled Health and Mana regeneration during Chrysalis. Stukov: Fixed an issue preventing Lurking Arm from revealing damaged targets if they were at the edge of its area of effect. Tyrande: Fixed an issue preventing Elune's Chosen from healing allies if damage was dealt to Shielded/Protected targets. Heir of the Raven Court is ready to descend upon the Nexus! Check out the official patch notes and our Orphea build guide to learn more.
    • By positiv2
      This thread is for comments about our Orphea guide for Heroes of the Storm.