I am going to ping @Spartan , @Triller, @Pifferfish, @Garsh, and @Banackock right off the bat for being around for this convo. Let's analyze a claim before taking a side on it.
Some of us here are no doubt familiar with the world of college football, and many of those here are also familiar with a little something called "the SEC" and some really funny things that happened to the SEC this year. For the great unwashed, the SEC is considered the most dominant conference in college football's recent history. With college football's playoff system determined by rankings, there is usually a controversy related to SEC teams when playoff season rolls around. Some SEC fans believe that their team deserves a bit of extra credit for being in the SEC and playing mostly other SEC teams, who they believe are better as a whole than the others. As some people may argue, a team with a few losses should still be ranked highly and make the playoff field if those losses are to other teams who are so good that one would expect them to happen regardless of how good the losing team is. The overwhelming majority of those who support non-SEC teams feel disrespected by this mindset held by some SEC fans and analysts, feel that it doesn't give their team enough credit, and let their rage be known on social media in advance of actually seeing the games play out and finding out just how good any one conference's teams really are.
This year, Alabama missed the playoff--something that seemed very unlikely under its new expanded format--and some people on the network who owns the SEC's TV rights lost their minds. Yes, worse teams than Alabama had made the playoff, but it was because Alabama had simply lost more times and thus was ranked lower. Those people just had to cry about it, and they did, with lots of excuses being made for Alabama and lots of promises that the SEC teams that had made the playoff would demolish the competition. After this point, it then got really funny when just about every SEC team in the playoff got flat-out embarrassed, and it also got really cool for me (consider my biased perspective on the above) when I watched my non-SEC team win the championship.
It reminded me a lot of something that I've seen way too much over my time in the VHL, which is "my team lost in the playoffs so now I have to melt down and demand that the structure of the league changes to my benefit"-itis. Now, I'm not saying at all that this is what happened on VHL Discord tonight. All that happened was that a claim was made, and to the best of my efforts to represent it in an unbiased manner, it is as follows:
STHS provides a morale boost to winners of wild-card rounds in the playoffs, which matches them up better than they deserve with the higher-seeded team in the next round.
Because of this, the wild-card winner beats the higher-seeded team more often than they deserve. I'm told that @Advantage or @CowboyinAmerica or someone has done this analysis in the past, but I'm not immediately familiar with it.
Since the wild-card team has an unfair advantage and consistently beats the higher-seeded team, we should re-evaluate how the playoff system works so as to avoid this unfair advantage.
This is all well and good. I'd completely agree that if it were the case that the wild-card team were winning upset matchups over top seeds at an unfair rate, then mitigating that would be a proper course of action. But still, a claim was made and I don't see any numbers--and I've spent the past years of my life trying to learn exactly when, where, and how to dispute stuff that other people write.
To resolve this, I skip the gym and stay up late tonight so I can get to the bottom of this whole thing. To me, it shouldn't be all that difficult to look at the whole morale part. Just wait until next season rolls around and make sure you pop open the index to find out whether that bump from the first round is really there. If there really isn't any change, then the whole point is moot. And if there is one, is there a way to adjust other teams' morale manually to match? I don't know how simming works.
For now, though, let's find out whether it really is true that wild-card teams win too often.
I'D LIKE YOU TO HAVE A SPREADSHEET THAT I'M PROUD OF.
Start on the first tab, which is an overview of every wild-card winner from the past 10 seasons, compared to the teams they then faced in the next round of the playoffs. On the surface, the argument that these teams win disproportionately seems reasonable. After all, wild-card winners have a winning record of 11-9 against top seeds over this time. Clearly, this isn't what should be considered likely or expected.
But is it actually outside the limits of what we would expect, or is this explainable by parity?
It's important to note that my model is based on one underlying assumption, and that is that it uses regular-season points as a proxy for overall team ability and the likelihood that a team will win any given game. If you don't agree with this, then you don't agree with any of my analysis. I'm not sure there's a better way to do it, though--it's based on proven history over the regular season, and there's the added bonus that those teams had played each other a handful of times on the way to getting those points. Based on this, S88's Vancouver (with 77 points) would have about a 43% chance of winning any one game against Seattle (102 points). One would obviously expect Seattle to win that playoff series--and they did--but there's a nonzero chance that Vancouver would have pulled the upset. About a 33% chance, actually, according to my estimates.
Here's how I did that: I took the single-game winning percentage and generated 1000 random numbers with it. If the number were below 430 (out of 1000), I counted that as a win for Vancouver, and if it were above 430, I counted it as a win for Seattle. That gave me 1000 simply simulated games between those two teams, from which I used a separate list to track totals. Every time one team reached 4 wins before the other, the numbers would reset and it would count as a simulated series win for the team in question. As it turns out, again, based on this method, Vancouver would pull off the upset about a third of the time. This is a very simplified example, but it's nonetheless an example of something called Monte Carlo simulation, where models are drawn up based on random generation.
I did this for every wild-card winner's matchup with the top seed of their conference, digging up simulated probabilities that were pretty low for matchups like S90's London versus Davos (unsurprisingly won by Davos) and pretty high for the same season's Vancouver versus Toronto. That one, as of right now (all the random generation regenerates every time something changes with the sheet), says that underdog Vancouver actually wins more of the time than Toronto--which I'm OK with. Random simulation is random and doesn't always match what we would expect; as long as it's in reason (and the difference is only slight in this case), that's OK.
Obviously, in almost every run, the top seed wins more simulated series than the wild-card team, so we would expect the average number of series won by those top seeds to be greater. I can change numbers on the spreadsheet, but the cumulative win total of all wild-card teams hovers pretty closely around 7.8. That doesn't change much at all--according to my model, if the VHL playoffs somehow managed to have the same sort of huge sample size that my model does, wild-card teams would have a record of about 8-12 over the past 10 seasons. We knew this already, but they have been winning more than expected. What we don't know is whether this actually means anything.
So, we dig back into the same random dataset that we used before. Conveniently, we have the probability that a team would have won a series (there's an important distinction between winning a game versus stringing together 4 wins out of potentially 7, of course, but we've already accounted for this). Taking S91's DC versus LA as an example, my model says that DC would have won this series 57 times and LA would have won it 116 times over the 1000 games that I simulated. So, all we have to do is to generate a random number between 1 and 173 (that's just 57+116), and if that number is above 57, it's a win for LA, and if it's below, it's a win for DC. We do this for every playoff series.
Thankfully, the sheet changes all its random cells every time I do anything with it, so every entry is a new simulation of every single playoff series. Barring the time it takes for Google to come up with 20,000+ random numbers at a time, this is great--all I had to do at this point was to write down the total number of wins by wild-card teams, wait a handful of seconds, and have my next number ready to go.
I simulated 21 different outcomes (to give 20 statistical degrees of freedom), and this gave me a big range of success (and lack thereof) for wild-card teams. Over the course of these runs, wild-card teams put up a record as bad as 4-16 and as good as 12-8 (better than actual history!) in my simulations. 11 wins was even matched twice. Something that makes me feel really good about the accuracy of these runs was that the average of series wins here was also 7.8--pretty much exactly the same as the number I reach when I add the cumulative winning percentages. It's another random simulation that matches the first one pretty independently of it, so I really feel confident that it's describing the state of affairs accurately.
Based on these outcomes, I now had an average and a standard deviation, which I could finally use to do something you probably did in high school: a simple t-test for probability. For those unfamiliar, the t-test is a statistical method that's used to calculate the probability that a given data set could have been generated by chance. Generally, p-values below 0.05 are considered "statistically significant" and good reason to reject the null hypothesis. In stat-speak, that means:
Null hypothesis: wild-card teams do not win playoff series at a disproportionately high rate.
Alternate hypothesis: wild-card teams do win playoff series at a disproportionately high rate.
If the t-test gives us a number below 0.05, then we can reject the null hypothesis. This does not necessarily mean that we accept the alternate hypothesis (which is important because it's the reason why we do this to begin with)--it just means that we conclude that the statement I've listed as the null hypothesis is untrue.
However--taking the average, standard deviation, and sample size, and considering how far 11 wins is from our average of 7.8, we get a p-value of:
This means that at the moment, based on the information I've come up with, I cannot in good conscience agree with a claim that wild-card teams are afforded an unfair advantage in the second round of the playoffs. There are things that can change this. Perhaps if I go farther back in time, I end up seeing more wild-card wins that push the historical average farther away from the simulated one. Perhaps I simulate another 20 rounds and get a more comprehensive distribution that makes 11 wins look worse (although I wouldn't count on that based on my averages matching and 11 wins appearing to be well within the range of variability). It is not an incorrect statement that wild-card teams have won more often than expected in recent seasons. But, if it were proposed that we change the playoff format because wild-card teams are finding success, I would not currently support it because I can't reasonably say that they're finding it unfairly. I had no idea what the outcome of this analysis would be when I started it, and if I'd come across more significant results, I would have gotten fully behind that idea.
So, with apologies to the Moscows and the Malmos and the Vancouvers of the world, sometimes these things happen and that's OK. I hope it can at least be respected, whether or not you agree with me, that I make my case only after having done my best to back it up.
.png.ed69a7260a477fcf4feead7a0e4b7506.png)
I have gone through a few different iterations of updating my guide and I forgot that it still has a 200-TPE cap on suggestions for the M. Maybe I’ll have to go through it again. This looks good though! We’ll always take cool resources for M players.
