wow An Important Word on Logs and DPS Rankings

[Starym 3,776]

 

We've been taking a look at WarcraftLogs and their overview of the weekly DPS rankings in the pre-patch recently, so we wanted to clarify the relevance of some of the data and its impact. We asked two of our top guide writers to talk to us about the meaningfulness of the data and how it may be skewed within the pre-patch environment (and in Shadowlands), so here are Wordup and Impakt giving us the details!

Wordup on spec representation affecting the data

Quote

When it comes to pre-patch not all log statistics are created equal, and the quality of the data you get is usually decided upon how popular the spec is out of the gate. Those that have high play counts tend to naturally smooth out errant pulls due to the sheer volume of data, but those with a lower playerbase can sometimes get polluted by a few things, given there are only one or two weeks of logs to pull from. Most commonly players dying due to bugs, server issues or people being brought in for paid boosts can warp the averages a lot for low population specs in a pre-patch environment.

Pre-patch statistics are also fairly limited to the final raid of the expansion (in this case Ny'alotha Mythic). This is due to lower difficulties or older raids simply not being ran by the majority of the endgame playerbase on an equal playing field (be that gear or experience discrepancies) so results are immediately polluted by extremes.

When a spec has only a small number of logs to generate these averages, you tend to see very high skews in top to bottom performance when you manually look at each percentile bracket. For something like Ny'alotha, a rough estimate of anything above the 3000 logged fights is probably a good metric to assume data is starting to coalesce (250 logs per each of the 12 fights gives room for a good data set to be generated for even late encounters). The further you pinch that number, the less accurate the results will become, and the more extreme the deviations from odd data will be when you look at overall statistics. Specs like Survival, Feral or Demonology are not as easy to gauge relative to say Beast Mastery or Shadow, because they simply don't have the dataset generated from so few kills to concretely iron out external factors and smooth out the player skill curve.

As you can see from the above, the representation of everything under Arms Warrior might be a little suspect, especially when it comes for Survival, Demonology, Frost DK, Feral and Fire Mage.

Impakt on the relevance of the above for Shadowlands.

Quote

This trend of insufficient data for underrepresented specs is not even unique to pre-patch. During many patches, including in BfA, you will see the majority of players flock to the best classes and specs, which means significantly less data available for other specs, and things will often go unnoticed. Demonology is a great example as it has been quite strong for the majority of 8.3, but because the other Warlock specs were so much more popular and could consistently perform at a top level, it is one of the least played specs.

In fact, the "BiS" talent combinations, Azerite choices, and Essences for many specs have drastically changed over the course of the expansion, not necessarily because the spec itself changed, but because often players will mimic the accepted meta, leading to off-brand choices never seeing play until enough people try it out for themselves.

For instance, 4-5 months ago the accepted meta for Destruction Warlock changed almost overnight when a few Warlocks started getting incredible parses using Eradication and Internal Combustion. Over the next few weeks the Warcraft log rankings completely shifted to Destruction Warlocks exclusively using this newer talent combination. Even the most popular specs will sometimes see new builds emerge from the innovation of handfuls of players.

And so while the DPS logs are certainly interesting and fun to look at and talk about, they aren't the be-all and end-all of the real spec situation, as there are many more factors to examine. We'll be posting the parse counts for each spec in our weekly overviews going forward as well, so players can have a better overview of the situation - but always keep in mind: if you're having fun with a spec, just keep at it and keep experimenting, nothing is set in stone!