Jump to content
FORUMS
Stan

Comments on Performance-Based Matchmaking by Lead Systems Designer Travis McGeathy

Recommended Posts

MSKG7EUORNAF1456364705258.jpg

Redditor Elitesparkle gathered insightful comments by Lead Systems Designer Travis McGeathy about the upcoming performance-based matchmaking system.

Heroes of the Storm will be soon updated with a new MMR system that takes individual gameplay into account. The current system is limited to wining / losing a game. The new system is more accurate, because it uses various parameters to calculate your MMR. Lead Systems Designer Travis McGeathy answered some interesting questions about the new system over on reddit.

Screenshots from the What's Next Panel

The 2018 update is scheduled for mid-December.

CPcfp1u.jpgUTdXHOA.jpgsN0oZWC.jpg

Blizzard LogoBlizzard (Source)

Any changes also planned for uncertainty?

Uncertainty doesn't exist in the same form in the new system. The core concept of something that limits how much MMR can adjust on a per-game basis is there, but the factors that go into it are different.

Looks like new system promotes maining a few heroes rather than flexibility, true or not, and how big is the impact if true?

The most important thing is still winning the game. That hasn't changed. Sometimes, that means sticking to heroes you know really well. Other times, that means having flexibility in your hero roster and playing the heroes that best fit the situation.

Might be a stupid question, but if it's only about winning, why do we have stats-based MMR? why the win isn't the only factor? Isn't the concept of stats-based MMR less accurate than the winrate itself to calculate your odd of winning?

It's about speed more than accuracy. The current system, based entirely on wins vs loses, works, but because your skill is diluted in a team of 5, unless you are dramatically better (or worse) than the rest of the team, it can take a large number of games before the difference in skill translates to more wins (or loses) and moves your MMR to where it should be. By factoring in individual skill to the equation, it can arrive at the proper MMR much faster which means the matchmaker can make better matches overall.

Is draft/banning taken into account? Choosing the right hero in draft may be more important than performing better than peers on a certain hero.

Drafting matters in the same way as the above statement. Drafting is a skill and how well you draft affects your win rate.

All in all, do you have some additional insight into what parameters are used? How zoning, peeling, bodyblocking, skillshot blocking is taken into account? How will Force Wall-Tassadar be able to compete with Archon-Tassadar?

We have great insight into what parameters are used...but aren't going into that level of detail externally.

Also do you plan to add more parameters in the mix?

We'll add more parameters as it makes sense.

There's some concern about how reliable is data on low played heroes — maybe like Chen, Rexxar, TLV, anything on that? (Though I can't imagine any answer here but "we think our data is reliable enough" :D)

We have more than enough data on all existing heroes. For new heroes or major reworks, the performance-based adjustments will basically be disabled until enough data exists.

Are we right to assume that new MMR calculation would be applied to all game modes? For QM, it looks like a step in the direction of per-hero MMR, do you think we may end up there, or at least do you think it may be viable to maintain?

It will apply to all game modes that have MMR.

Can you win a game yet still lose MMR points? IE: a player goes full tilt, intentionally tries to throw. But the other 4 players are just dominating and still win the game.

No. You'll only gain points on a win and lose them on a loss. If you get full carried for a game, you won't lose points, but you also aren't going to gain very many either.

I think people's main concerns are along the lines of "who determines which stats are important" and "why these stats and not these other ones". I assume you guys are running some kind of adaptive machine learning, but most people are not going to understand that so maybe some kind of brief summary into how that works can help to alleviate the main concerns. Just a suggestion.

Quite right. For the "who determines which stats are important", its the players. We've chosen the stats to monitor, but which ones are important for that situation is something we are measuring, not determining ourselves.

About revealing MMR and the general clarity. Assuming "no" on "if you're going to reveal MMR" (:D) I'd like to ask whether we would be able to see when the adjustment takes place on the post-match screen, like we see PRA now.

We won't have visible MMR for this update, but I do still want to do it in the future. For now, you'll mainly see the effects through rank points. We're planning to swap out personal rank adjustment for a more direct performance adjustment which will give better insight while filling a similar role of keeping rank close to MMR. I'm not sure right now whether that will be for this season or the next yet.

Spot on and part of why visible MMR hasn't been a big priority for us but, there are some advantages to it:
- Currently, rank is the only visible indicator of skill. This forces us to link rank to MMR so they don't diverge too much because players get upset when they see people of different rank in their games even if they are actually the same skill. That's not ideal and leads to oddities with matchmaking when rank and MMR do diverge.
- Even though we don't show MMR, players have created other sources to approximate their MMR. The information isn't very accurate, but being the only source of knowledge, it gets cited frequently when players feel there are issues with matchmaking. There's two issues here: first, it gives a false impression that matchmaking is poor for some players and second, it makes it harder to understand when there are actually issues with matchmaking since the real issues can get buried. Effectively, for players that really care about MMR, we already have most of the downsides associated with visible MMR without having the upsides of it being accurate information.

But what order of magnitude are you all thinking? 10 points? 20, 50? Just trying to get a rough understanding of the overall impact it might have on gaining.

That's a tuning point we haven't settled on yet.

The system, unless specifically programmed to do otherwise, will think a death at the last moment is extremely meaningful since it is normally a 60s death timer.

Time is factored into the measurements. If you're dead for the last few seconds of a match, there wouldn't be any measurable impact relative to everything else that happened that game.

Regarding the new performance based MMR changes, does this mean if my team has captured the winning altar on Towers of Doom and for the fun of it I walk into the enemy death zone and die, my MMR will be adversely affected from this little bit of fun? Or perhaps there are 5 enemies killing my core and there are 4 allies dead. I have no chance to defend the core but rather than try and risk receiving an extra death I won't lose as much MMR if I stay in base? Curious about this.

Everything is by time, including deaths. If you die in the last second of a match, its not going to mean much. Even a full death in the last minutes would really only be a minor modifier overall once diluted against your overall performance in the game. The biggest factor for MMR adjustment is still whether you won or lost. Your overall performance is secondary to that and any particular moment is just a small part of that overall performance.

Does this mean that winning a game quickly and playing well will reward the same personal adjustment as having good stats in a long game? Some people are concerned that players will want to artificially extend game lengths to try and pad their stats before winning. Especially in games where the opponent is greatly outmatched.

Yeah, definitely. Everything factors game time into it for that exact reason.

Share this post


Link to post
Share on other sites
Uncommon Patron

My conclusions:

-Win ratio is the most important

-Game lenght does not affect performance

-It might promote the usage of the same hero(he tried not to give a clear answear so this means the answear wouldn't please the players)

-We still don't know how MMR is calculated.

Share this post


Link to post
Share on other sites
11 hours ago, Stan said:

The most important thing is still winning the game. That hasn't changed. Sometimes, that means sticking to heroes you know really well. Other times, that means having flexibility in your hero roster and playing the heroes that best fit the situation.

That will force people to actually learn the Heroes in Try Mode or VS AI before compromising entire team compositions because they don't know how things work. For instance, we all know that "Nova main" who doesn't even know that the red eye icon means you've been detected.

Share this post


Link to post
Share on other sites

I was hoping new system would go away from winning being only major parameter, because it's team statistic, not something that is entirely upon you.

I'll only mention one other system from I game I used to play, WN8 for World of Tanks.
This measure is community driven, nothing official. Despite being system, well, that calls out for  stat-whoring people (don't know other word), it's really hard to do so that you aren't actively winning. Main component to WN8 was doing damage and assisting others in doing damage, only way to break system was to be able to somehow do a lot of damage on game that will be lost, really hard to do. 
Winrate is part of it, but it only made a small portion.

I'm not sure why are they persistent on winning being main driver of what happens with your MMR.
 

Can you win a game yet still lose MMR points? IE: a player goes full tilt, intentionally tries to throw. But the other 4 players are just dominating and still win the game.

No. You'll only gain points on a win and lose them on a loss. If you get full carried for a game, you won't lose points, but you also aren't going to gain very many either."

They stated it themselves, it's still not accurate in those extremities.

Edited by SleepySheepy

Share this post


Link to post
Share on other sites
2 hours ago, SleepySheepy said:

I was hoping new system would go away from winning being only major parameter, because it's team statistic, not something that is entirely upon you.

I'll only mention one other system from I game I used to play, WN8 for World of Tanks.
This measure is community driven, nothing official. Despite being system, well, that calls out for  stat-whoring people (don't know other word), it's really hard to do so that you aren't actively winning. Main component to WN8 was doing damage and assisting others in doing damage, only way to break system was to be able to somehow do a lot of damage on game that will be lost, really hard to do. 
Winrate is part of it, but it only made a small portion.

I'm not sure why are they persistent on winning being main driver of what happens with your MMR.

The main difference is that World of Tanks has 15v15 matches, where teamwork is not as crucial as in HotS (that's not to say that it is not important in WoT), and since there is no draft phase, you don't have to be able to adapt to enemy team's composition. You can simply roll with a Tier X SPG and get a high rating. If people were to do that in HotS, the team compositions would be pretty bad, so, as they said, those are reflected in the win rate. 

Share this post


Link to post
Share on other sites
4 hours ago, SleepySheepy said:

I was hoping new system would go away from winning being only major parameter...
 

Can you win a game yet still lose MMR points? IE: a player goes full tilt, intentionally tries to throw. But the other 4 players are just dominating and still win the game.

No. You'll only gain points on a win and lose them on a loss. If you get full carried for a game, you won't lose points, but you also aren't going to gain very many either."

They stated it themselves, it's still not accurate in those extremities.

And they did. Now it's not the only major parameter, but one of many. But it's still a priority, which is very logical, isn't it? Or you were hoping that even when losing a match you could still EARN mmr? I'm sorry but I couldn't help reading that between your lines.

And just skipped the rest of your post about WoT. It's completely irrelevant simply because comparing games of the same genre is a risk of running into wrong argumentation, let alone mentioning games of completely different nature.

Share this post


Link to post
Share on other sites

I understand winning and losing being the primary parameters.  If they weren't, EVERYONE would just prioritize Nova and try and go 15-0 every match, losses be damned.

Speaking as someone who hovers around a 50% win rate (keeping in mind I play my lowest ranked hero to fill quests), I have a lot of games with people who, frankly, have no idea about even basic strategy.  I also go on a ton of long (10 games+) loosing streaks.  And I almost never carry.  HotS has been VERY frustrating at times as a result.  As far as I'm concerned, MMR matchmaking can't get much worse then it currently is.

Share this post


Link to post
Share on other sites
15 hours ago, gamerk2 said:

I understand winning and losing being the primary parameters.  If they weren't, EVERYONE would just prioritize Nova and try and go 15-0 every match, losses be damned.

Speaking as someone who hovers around a 50% win rate (keeping in mind I play my lowest ranked hero to fill quests), I have a lot of games with people who, frankly, have no idea about even basic strategy.  I also go on a ton of long (10 games+) loosing streaks.  And I almost never carry.  HotS has been VERY frustrating at times as a result.  As far as I'm concerned, MMR matchmaking can't get much worse then it currently is.

Yep, matchmaking in HotS is terrible, that's why I quit. Even though I pretty much always had the best stats (xp, damage, ka/d ratio, heal) in lost matches, this system is not making me coming back.

Enough is enough.

Edited by Esmer

Share this post


Link to post
Share on other sites
Quote

 I assume you guys are running some kind of adaptive machine learning, but most people are not going to understand that so maybe some kind of brief summary into how that works can help to alleviate the main concerns. Just a suggestion.

Why people are so afraid to ask the damn question ? That question is: when will you reveal what is taken into consideration when calculating one's MMR ?

If Blizzard want HotS to be a competetive game then how players are supposed to be competetive when they don;t know how system works ?? This irritates me so much :/ If the system is as good as they describe it then there should be no problem with telling people how does it work exactly. If they don't want to reveal this info then there is sth wrong with it imo.

Share this post


Link to post
Share on other sites

I hope HotS dies out as a result of the poor matchmaking that they've had since the start. Came over to see the pretty models left after hitting master q'ing as Aba/Murky. This game is the pretty version of a League of Legends shortbus.

Share this post


Link to post
Share on other sites
On 14. 11. 2017. at 5:21 PM, Jonar said:

And they did. Now it's not the only major parameter, but one of many. But it's still a priority, which is very logical, isn't it? Or you were hoping that even when losing a match you could still EARN mmr? I'm sorry but I couldn't help reading that between your lines.

And just skipped the rest of your post about WoT. It's completely irrelevant simply because comparing games of the same genre is a risk of running into wrong argumentation, let alone mentioning games of completely different nature.

Yeah, you "read" it right. Seems silly to me that MMR will be positive or negative just depending on win/loss.

Is that wrong thinking? I would say it's not, but of course, I could be wrong.

  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Stan
      Braxis Outpost, the single-lane Starcraft-themed battleground, is available as this week's brawl. It comes with standard ARAM rules, which include disabled Hearthstones and no healing at spawn. Complete 3 matches of Braxis Outpost to earn a Loot Chest.
      Blizzard (Source)
      This week’s Heroes Brawl is Braxis Outpost! A single-lane battleground set in the Koprulu Sector, it will be up to your team to survive in the lane and take down the enemy core, without the aid of Hearthstone or their Hall of Storms. Good luck, have fun!
      Rules:
      Shuffle Pick Choose from one of three randomly selected Heroes before entering the battle on Braxis Outpost. Be quick about it though, you only have 30 seconds to choose! ARAM Rules Players cannot use Hearthstone Players do not heal at their starting location. (Hall of Storms) Cores have no shields Mercenary Camps Braxis Outpost features two mirrored Goliath & Raven mercenary camps. Be the first team to take down the enemy Core to claim victory! Rewards:
      Complete three matches of Braxis Outpost to earn a Loot Chest! Find out more about the Heroes Brawl game mode on our Heroes Brawl site; and as always, you can find more information on this week’s Brawl by clicking the Brawl Info button at the bottom of the play screen when preparing to queue for the Brawl game mode.
    • By Stan
      Qhira now costs 10,000 Gold. This week's Heroic Deals include Junkrat, Kel'Thuzad, and The Lost Vikings.
      Click here to check out this week's Free-to-Play Hero rotation.
      Hero Sales
      Heroes Old Price New Price Junkrat 750 Gems 375 Gems Kel'Thuzad 750 Gems 375 Gems The Lost Vikings 750 Gems 375 Gems Skins
      Deathranger Nova Bundle can still be purchased for 1,500 Gems and includes:
      Deathranger Nova Mirage Deathranger Nova Rugged Deathranger Nova Ember Deathranger Nova Dusk Deathranger Nova
      Mounts
      3 mounts have been added to the game and they cost 10,000 Gold each.
      Scavenger's Ripper Outrider's Ripper Soldier's Ripper
    • By Stan
      This week's rotation includes Brightwing, Kel'Thuzad, Medivh, and more.
      Free-to-Play Hero Rotation: August 20, 2019
      The required player level remains unknown because Blizzard stopped updating the official forum post, so we're listing the Heroes alphabetically.
      Azmodan Brightwing Jaina Kel'Thuzad Medivh Muradin Nazeebo Raynor Rehgar Rexxar Samuro Sonya Uther Valeera (Source)
    • By Stan
      In Storm League, rank is always updated by taking the change in rating and determining the equivalent number of rank points to reward. While the Heroes team is quite happy with how it turned out, the system is causing problems in Bronze 5.
      In Bronze 5, players gain/lose far fewer points per game than players in other leagues and divisions. The underlying problem makes it harder to climb out of Bronze 5 and causes frustration and bad gameplay experience. The team is working on ways to improve it and provided an explanation of why this is happening.
      Blizzard (Source)
      With the launch of Storm League season 1 we have seen a dramatic improvement in rank distribution. One of the primary changes in this release was in how rank is determined. In the past, rank and rating were two entirely different concepts, both of which were updated differently after each game. It was very common for a player's rank and rating to diverge throughout each season which could cause the matchmaking system to struggle to find games.
      With the changes introduced this season, rank is always updated by taking the change in rating and determining the equivalent number of rank points to reward. This leaves very few ways rating and rank can diverge and results in most players being shown a number equivalent to their rating.
      Overall, we are very happy with how the changes are looking; however, we are aware of the inconsistencies players are currently experiencing in Bronze 5. With the new design, players in Bronze 5 will gain/lose far fewer points per game than players in all of the other leagues and divisions. The reason for this is a bit technical but we wanted to take the opportunity to elaborate on what's going on.
      Under the hood, matchmaking rating is simply a number on a bell curve. Every league and division encompasses a range of values on the same curve. For example, if the overall bell curve ranges from -10.0 to 10.0 and the range 1.0 to 1.5 maps to Gold 1, then a player with a rating of 1.25 will have the equivalent rank of Gold 1 & 500/1000 points. These numbers are intentionally made up as the real ones are always subject to change but the idea is the same.
      The reason for the decrease in points awarded in Bronze 5 is that the range of ratings that maps to Bronze 5 is much larger than other divisions. The difference is that this much larger range of ratings still maps to the same number of rank points. As a result, despite being rewarded the same change in matchmaking rating, the rank points awarded may be lower. This is visualized by the graph below:
      We understand that this inconsistency is a negative experience for players so we will be exploring some ways to improve it.
      Thanks for participating in Storm League and providing your feedback!
    • By Stan
      Industrial District is this week's brawl. The single-lane Overwatch-themed map comes with standard play and shuffle pick. The main goal is to destroy the enemy Core and playing three games rewards a Loot Chest.
      Blizzard (Source)
      This week’s Heroes Brawl is Industrial District! Slide around and slug it out in the heart of Volskaya in this Overwatch-themed, single-lane Battleground featuring lots of conveyor belts and tons of action.
      Rules:
      Shuffle pick - Choose from one of three Heroes before entering the battle. Standard play - no talent or level restrictions. The first team to destroy the enemy Core wins! Rewards:
      Complete three matches of the Industrial District Brawl to earn a Loot Chest! Find out more about the Heroes Brawl game mode on our Heroes Brawl site; and as always, you can find more information on this week’s Brawl by clicking the Brawl Info button at the bottom of the play screen when preparing to queue for the Brawl game mode.
×
×
  • Create New...