I Settle a Debate I Wasn't Part Of By Being a Huge Nerd: Are #1 Teams Really Being Screwed Over?

[Gustav 6,575]

 

I am going to ping @Spartan , @Triller, @Pifferfish, @Garsh, and @Banackock right off the bat for being around for this convo. Let's analyze a claim before taking a side on it.

 

 

Some of us here are no doubt familiar with the world of college football, and many of those here are also familiar with a little something called "the SEC" and some really funny things that happened to the SEC this year. For the great unwashed, the SEC is considered the most dominant conference in college football's recent history. With college football's playoff system determined by rankings, there is usually a controversy related to SEC teams when playoff season rolls around. Some SEC fans believe that their team deserves a bit of extra credit for being in the SEC and playing mostly other SEC teams, who they believe are better as a whole than the others. As some people may argue, a team with a few losses should still be ranked highly and make the playoff field if those losses are to other teams who are so good that one would expect them to happen regardless of how good the losing team is. The overwhelming majority of those who support non-SEC teams feel disrespected by this mindset held by some SEC fans and analysts, feel that it doesn't give their team enough credit, and let their rage be known on social media in advance of actually seeing the games play out and finding out just how good any one conference's teams really are. 

 

This year, Alabama missed the playoff--something that seemed very unlikely under its new expanded format--and some people on the network who owns the SEC's TV rights lost their minds. Yes, worse teams than Alabama had made the playoff, but it was because Alabama had simply lost more times and thus was ranked lower. Those people just had to cry about it, and they did, with lots of excuses being made for Alabama and lots of promises that the SEC teams that had made the playoff would demolish the competition. After this point, it then got really funny when just about every SEC team in the playoff got flat-out embarrassed, and it also got really cool for me (consider my biased perspective on the above) when I watched my non-SEC team win the championship.

 

It reminded me a lot of something that I've seen way too much over my time in the VHL, which is "my team lost in the playoffs so now I have to melt down and demand that the structure of the league changes to my benefit"-itis. Now, I'm not saying at all that this is what happened on VHL Discord tonight. All that happened was that a claim was made, and to the best of my efforts to represent it in an unbiased manner, it is as follows:

 

• STHS provides a morale boost to winners of wild-card rounds in the playoffs, which matches them up better than they deserve with the higher-seeded team in the next round.
• Because of this, the wild-card winner beats the higher-seeded team more often than they deserve. I'm told that @Advantage or @CowboyinAmerica or someone has done this analysis in the past, but I'm not immediately familiar with it.
• Since the wild-card team has an unfair advantage and consistently beats the higher-seeded team, we should re-evaluate how the playoff system works so as to avoid this unfair advantage. 

 

 

This is all well and good. I'd completely agree that if it were the case that the wild-card team were winning upset matchups over top seeds at an unfair rate, then mitigating that would be a proper course of action. But still, a claim was made and I don't see any numbers--and I've spent the past years of my life trying to learn exactly when, where, and how to dispute stuff that other people write. 

 

To resolve this, I skip the gym and stay up late tonight so I can get to the bottom of this whole thing. To me, it shouldn't be all that difficult to look at the whole morale part. Just wait until next season rolls around and make sure you pop open the index to find out whether that bump from the first round is really there. If there really isn't any change, then the whole point is moot. And if there is one, is there a way to adjust other teams' morale manually to match? I don't know how simming works.

 

For now, though, let's find out whether it really is true that wild-card teams win too often.

 

 

I'D LIKE YOU TO HAVE A SPREADSHEET THAT I'M PROUD OF.

 

 

Start on the first tab, which is an overview of every wild-card winner from the past 10 seasons, compared to the teams they then faced in the next round of the playoffs. On the surface, the argument that these teams win disproportionately seems reasonable. After all, wild-card winners have a winning record of 11-9 against top seeds over this time. Clearly, this isn't what should be considered likely or expected.

 

But is it actually outside the limits of what we would expect, or is this explainable by parity?

 

It's important to note that my model is based on one underlying assumption, and that is that it uses regular-season points as a proxy for overall team ability and the likelihood that a team will win any given game. If you don't agree with this, then you don't agree with any of my analysis. I'm not sure there's a better way to do it, though--it's based on proven history over the regular season, and there's the added bonus that those teams had played each other a handful of times on the way to getting those points. Based on this, S88's Vancouver (with 77 points) would have about a 43% chance of winning any one game against Seattle (102 points). One would obviously expect Seattle to win that playoff series--and they did--but there's a nonzero chance that Vancouver would have pulled the upset. About a 33% chance, actually, according to my estimates.

 

Here's how I did that: I took the single-game winning percentage and generated 1000 random numbers with it. If the number were below 430 (out of 1000), I counted that as a win for Vancouver, and if it were above 430, I counted it as a win for Seattle. That gave me 1000 simply simulated games between those two teams, from which I used a separate list to track totals. Every time one team reached 4 wins before the other, the numbers would reset and it would count as a simulated series win for the team in question. As it turns out, again, based on this method, Vancouver would pull off the upset about a third of the time. This is a very simplified example, but it's nonetheless an example of something called Monte Carlo simulation, where models are drawn up based on random generation. 

 

I did this for every wild-card winner's matchup with the top seed of their conference, digging up simulated probabilities that were pretty low for matchups like S90's London versus Davos (unsurprisingly won by Davos) and pretty high for the same season's Vancouver versus Toronto. That one, as of right now (all the random generation regenerates every time something changes with the sheet), says that underdog Vancouver actually wins more of the time than Toronto--which I'm OK with. Random simulation is random and doesn't always match what we would expect; as long as it's in reason (and the difference is only slight in this case), that's OK.

 

Obviously, in almost every run, the top seed wins more simulated series than the wild-card team, so we would expect the average number of series won by those top seeds to be greater. I can change numbers on the spreadsheet, but the cumulative win total of all wild-card teams hovers pretty closely around 7.8. That doesn't change much at all--according to my model, if the VHL playoffs somehow managed to have the same sort of huge sample size that my model does, wild-card teams would have a record of about 8-12 over the past 10 seasons. We knew this already, but they have been winning more than expected. What we don't know is whether this actually means anything.

 

So, we dig back into the same random dataset that we used before. Conveniently, we have the probability that a team would have won a series (there's an important distinction between winning a game versus stringing together 4 wins out of potentially 7, of course, but we've already accounted for this). Taking S91's DC versus LA as an example, my model says that DC would have won this series 57 times and LA would have won it 116 times over the 1000 games that I simulated. So, all we have to do is to generate a random number between 1 and 173 (that's just 57+116), and if that number is above 57, it's a win for LA, and if it's below, it's a win for DC. We do this for every playoff series.

 

Thankfully, the sheet changes all its random cells every time I do anything with it, so every entry is a new simulation of every single playoff series. Barring the time it takes for Google to come up with 20,000+ random numbers at a time, this is great--all I had to do at this point was to write down the total number of wins by wild-card teams, wait a handful of seconds, and have my next number ready to go.

 

I simulated 21 different outcomes (to give 20 statistical degrees of freedom), and this gave me a big range of success (and lack thereof) for wild-card teams. Over the course of these runs, wild-card teams put up a record as bad as 4-16 and as good as 12-8 (better than actual history!) in my simulations. 11 wins was even matched twice. Something that makes me feel really good about the accuracy of these runs was that the average of series wins here was also 7.8--pretty much exactly the same as the number I reach when I add the cumulative winning percentages. It's another random simulation that matches the first one pretty independently of it, so I really feel confident that it's describing the state of affairs accurately.

 

Based on these outcomes, I now had an average and a standard deviation, which I could finally use to do something you probably did in high school: a simple t-test for probability. For those unfamiliar, the t-test is a statistical method that's used to calculate the probability that a given data set could have been generated by chance. Generally, p-values below 0.05 are considered "statistically significant" and good reason to reject the null hypothesis. In stat-speak, that means:

 

• Null hypothesis: wild-card teams do not win playoff series at a disproportionately high rate.
• Alternate hypothesis: wild-card teams do win playoff series at a disproportionately high rate.

 

If the t-test gives us a number below 0.05, then we can reject the null hypothesis. This does not necessarily mean that we accept the alternate hypothesis (which is important because it's the reason why we do this to begin with)--it just means that we conclude that the statement I've listed as the null hypothesis is untrue.

 

However--taking the average, standard deviation, and sample size, and considering how far 11 wins is from our average of 7.8, we get a p-value of:

 

Spoiler

0.1337

 

This means that at the moment, based on the information I've come up with, I cannot in good conscience agree with a claim that wild-card teams are afforded an unfair advantage in the second round of the playoffs. There are things that can change this. Perhaps if I go farther back in time, I end up seeing more wild-card wins that push the historical average farther away from the simulated one. Perhaps I simulate another 20 rounds and get a more comprehensive distribution that makes 11 wins look worse (although I wouldn't count on that based on my averages matching and 11 wins appearing to be well within the range of variability). It is not an incorrect statement that wild-card teams have won more often than expected in recent seasons. But, if it were proposed that we change the playoff format because wild-card teams are finding success, I would not currently support it because I can't reasonably say that they're finding it unfairly. I had no idea what the outcome of this analysis would be when I started it, and if I'd come across more significant results, I would have gotten fully behind that idea.

 

So, with apologies to the Moscows and the Malmos and the Vancouvers of the world, sometimes these things happen and that's OK. I hope it can at least be respected, whether or not you agree with me, that I make my case only after having done my best to back it up.

[Rin 184]

Should the NHL have the Hawks and Preds redo their series in case it was a big fluke?

 

No disrespect to the preds, I'm a firm believer that Nashville sweeping them is a huge fluke and robs the Hawks of truly accomplishing what their capable of. I've spent the last few days in pure disbelief and it just doesn't make sense to me. I've spent the entire regular season watching the Hawks play great hockey it's just not fair.

 

If the Hawks lose again I will face that the Preds deserved the win, but I am just 100% sure it was a fluke and does a big disservice to the Hawks and the NHL.

[Triller 1,721]

I like seeing the numbers, especially yours.  I was secretly hoping you would take a look into this and at least come to some kind of actionable conclusion if it did seem like Simon was coddling wildcard teams but in the end it seems to be just the way the cookie crumbles.  I accept the results of it all either way but it is fun for discussion.  The best teams don't always win but they especially hate to lose to teams they think don't deserve to win.

 

And to think, these series aren't even over yet.  Crunch me some numbers on coming back from a 3-1 series deficit!

[qripll 535]

50 minutes ago, Gustav said:

Since the wild-card team has an unfair advantage

What advantage are we talking about? 

[Victor 11,143]

No spreadsheet but same result.

 

 

 

[Spartan 4,648]

With even more professional evidence and effort (I searched two words with a filter) I say "nuh uh!"

 

 

 

[LucyXpher 1,766]

7 hours ago, Gustav said:

So, with apologies to the … Malmos … of the world, sometimes these things happen and that's OK.

Sometimes? 

Edited February 12 by LucyXpher

[Gustav 6,575]

6 hours ago, Victor said:

No spreadsheet but same result.

 

 

 

Yeah I think it’s also pretty telling that my article (unintentionally) cherry-picks one of the more skewed eras of the bunch and the stats still say we shouldn’t panic over it. 
 

39% of wild-card winners going on to beat the next team is actually really interesting because that’s exactly the percentage of 7.8 out of 20 that my model predicts twice. There’s definitely some level of coincidence in that, but I think it really backs up what I did with it. 

[tcookie 985]

I was actually curious about what the real numbers said about the "wildcard boost" ... it's thrown around a lot and certainly playing for Malmo I understand the feeling, but it's still just feeling. Never got around to digging into it myself but it's cool to see the actual numbers.

 

I think STHS gets a bad rap for what is actually realism sometimes. When a simulator gives us something like the Columbus over Tampa sweep in the NHL back in Tampa's 62-win season it's ripped on mercilessly, but that happened. I've got my issues with STHS with stuff like the implementation of PA and DI, but I think the actual game results it gives are ultimately pretty good.

Edited February 12 by tcookie

[Victor 11,143]

3 hours ago, Spartan said:

With even more professional evidence and effort (I searched two words with a filter) I say "nuh uh!"

 

 

 

I wonder if the true reason is the fact the underdog GM tries harder to find an edge because usually you have a losing regular season record against the Victory Cup team.

 

So the top seed is likelier to stick with what has worked and only react if they go down in the series, but the bottom seed will have already adjusted lines to counter the top seed. And that's enough to get them over the line.

 

Then it's the bottom seem that gets overconfident and doesn't change what's working and hence the winning percentage falls away in the next rounds. But the Victory Cup winners are the ones who get hit in the beginning.

[Nykonax 1,604]

9 hours ago, Gustav said:

Based on this, S88's Vancouver (with 77 points) would have about a 43% chance of winning any one game against Seattle (102 points). One would obviously expect Seattle to win that playoff series--and they did--but there's a nonzero chance that Vancouver would have pulled the upset. About a 33% chance, actually, according to my estimates.

 

Here's how I did that: I took the single-game winning percentage and generated 1000 random numbers with it. If the number were below 430 (out of 1000), I counted that as a win for Vancouver, and if it were above 430, I counted it as a win for Seattle. That gave me 1000 simply simulated games between those two teams, from which I used a separate list to track totals. Every time one team reached 4 wins before the other, the numbers would reset and it would count as a simulated series win for the team in question. As it turns out, again, based on this method, Vancouver would pull off the upset about a third of the time. This is a very simplified example, but it's nonetheless an example of something called Monte Carlo simulation, where models are drawn up based on random generation. 

Imagine monte-carlo'ing instead of just using a negative binomial distribution.

 

[Gustav 6,575]

8 minutes ago, Nykonax said:

Imagine monte-carlo'ing instead of just using a negative binomial distribution.

 

I still think of you as being 14 shut up

[Spartan 4,648]

19 minutes ago, Victor said:

I wonder if the true reason is the fact the underdog GM tries harder to find an edge because usually you have a losing regular season record against the Victory Cup team.

 

So the top seed is likelier to stick with what has worked and only react if they go down in the series, but the bottom seed will have already adjusted lines to counter the top seed. And that's enough to get them over the line.

 

Then it's the bottom seem that gets overconfident and doesn't change what's working and hence the winning percentage falls away in the next rounds. But the Victory Cup winners are the ones who get hit in the beginning.

It's possible I guess, I only changed lines on Davos once for G7 and once for Riga after the first sim too. I don't know how many GM's actually do the mind-games thing, I've only ever done it with @Banackock and very slightly with Alex? No one else that I've noticed does funky shit with lines, though I used to really hard counter Hedge's lines on Riga when he was GM but we still had a top team.

 

I guess it depends on if a team is doing line experiments mid-season to see what works best, then sets and forgets for the playoffs. Or conversely, you don't make changes often during the season and start to frantically try changes in the playoffs. I just don't think *that* many GMs are making moves like that to make it a 10/15/20 season potential issue of wild cards overperforming.

[Lemorse7 1,039]

people touch there lines??  Waitt ...... we can change lines ???

 

[Nykonax 1,604]

3 hours ago, Gustav said:

I still think of you as being 14 shut up

3rd year of a math degree and all i can do with it is making fun of you for generating 20,000 random numbers in a spreadsheet. Maybe I am still 14...

But unironically though I think (and you've concluded) morale is just cope. I didn't really read much into the statistics you've done but I also don't think it really matters. I think people in general (not just VHL but like humanity as a whole) just has a really bad understanding of randomness. Especially in the VHL, the sample sizes are so small that any randomness is going super rampant and also super memorable just by nature of there being stakes to it.

Intuitively I would say that the wildcard record is pretty well within like an interval of what would be expected. If wildcard teams were like 19-1 or something then it'd be surprising, but it being slightly in favour of wildcard teams is surprising for sure, but it's also not that insane when parity in the league makes playoff games decently close to a coinflip.

There's even arguments to be made that the #1 seed isn't actually the "true" #1 seed. For instance this season maybe Vancouver/Malmo had 5 lucky wins which propelled them to the #1 seed, but if you simmed 10,000 games instead of 72 maybe Calgary or Moscow are the actual "true" #1's, and if you take the "true" wildcard vs. the "true" #1 that record is much worse for the wildcard now. It's just such low sample throughout the season and playoffs.

It's pretty similar to people thinking meta is unbeatable just because they saw Vancouver threepeat with it. However most of those series were coinflips when simmed over large amounts of games, and it was a lower than 1/16 chance Vancouver actually threepeated. But we saw the 1/16 universe so the hybrid system was implemented. (sidenote meta was broken for reasons relating to salary cap and build cost than sths power).

I also don't think its a factor of changing lines or anything, cause most changes are going to be essentially neutral, nothing you do is going to boost your odds by like 10+%. From experience the only swaps that made improvements was like switching which line was 1st and 2nd but even then it wasn't a huge difference and sometimes made it worse.

Overall people are just bad with understanding randomness and morale isn't a system that is making wildcard teams OP.

Edited February 12 by Nykonax

[Gustav 6,575]

53 minutes ago, Nykonax said:

3rd year of a math degree and all i can do with it is making fun of you for generating 20,000 random numbers in a spreadsheet. Maybe I am still 14...

TBF I think it paints the picture more clearly/is more explainable to have actual results of a simulation to talk about. It’s also more fun for me to say I simmed each matchup for 175ish series than to just calculate the probability directly; we are a sim league after all. This is why engineering > math. 
 

Something that would have been even simpler to do would be to stay on Discord a little longer, because @LucyXpher talked about how Malmo’s morale was higher than Riga’s in this past series—so the idea that it’s a matter of some weird wild-card winner morale boost is completely false to begin with. 
 

Regardless of how you do it, something I think is interesting is that regular-season points seem to be a very good indicator of playoff success with my average outcomes matching Victor’s historical numbers pretty much exactly. With that being the case, I was also surprised to find out that most matchups are something like 60-40 in favor of the #1 team, and even the most mismatched ones are about 70-30. In general, I’d totally agree that people don’t get randomness and think the playoffs should be completely deterministic. I’ve seen far worse examples of “the league needs to change the way it works because I lost,” based on much less, going back to my first season here. It’s never going to cease to exist in the VHL, and it’s always going to be a little bit annoying, which is probably why I felt the need to analyze it before it turned into something bigger and weirder. 

[Nykonax 1,604]

4 minutes ago, Gustav said:

before it turned into something bigger and weirder. 

time to start the "Gustav works for big STHS Morale and is rigging his spreadsheet sims to push a pro-wildcard narrative" psyop.

[Gustav 6,575]

14 minutes ago, Nykonax said:

time to start the "Gustav works for big STHS Morale and is rigging his spreadsheet sims to push a pro-wildcard narrative" psyop.

If wildcard teams have an unfair advantage, then why did I never win a Cup with Davos? Curious. 

[Nykonax 1,604]

7 minutes ago, Gustav said:

If wildcard teams have an unfair advantage, then why did I never win a Cup with Davos? Curious. 

maybe because you couldn't even make the wildcard

[Banackock 8,313]

5 hours ago, Spartan said:

It's possible I guess, I only changed lines on Davos once for G7 and once for Riga after the first sim too. I don't know how many GM's actually do the mind-games thing, I've only ever done it with @Banackock and very slightly with Alex? No one else that I've noticed does funky shit with lines, though I used to really hard counter Hedge's lines on Riga when he was GM but we still had a top team.

 

I guess it depends on if a team is doing line experiments mid-season to see what works best, then sets and forgets for the playoffs. Or conversely, you don't make changes often during the season and start to frantically try changes in the playoffs. I just don't think *that* many GMs are making moves like that to make it a 10/15/20 season potential issue of wild cards overperforming.

I sometimes do funky and sneaky shit hehe and I certainly counter and search for patterns - but nowhere near like or as often I used to do.

[Rin 184]

This is slightly off-topic, but I want to say thank you to @Gustav  for taking this from Discord into a forum post and putting so much effort into backing an argument. Not only do I get to read through something I would have entirely missed otherwise, I got to see discussion added over time and even contribute if I wanted to (I'm not nearly knowledgeable enough to get into it though).

 

Genuine stick taps all around, very good thread that is curing some work time boredom.

[Garsh 1,201]

7 hours ago, Nykonax said:

Imagine monte-carlo'ing instead of just using a negative binomial distribution.

 

 

 

[Spartan 4,648]

2 hours ago, Gustav said:

If wildcard teams have an unfair advantage, then why did I never win a Cup with Davos? Curious. 

Huge self snitch lmfaoooo 

[Gustav 6,575]

2 minutes ago, Spartan said:

Huge self snitch lmfaoooo 

 

I like to think I own it well at this point.

[eaglesfan036 4,615]

This was a good article guatav