- MockAnalysisIsMyDrug
- Posts : 219
Reputation : 247
Join date : 2019-11-19
2023 Regionals By The Numbers
Sun Nov 27, 2022 10:22 pm
At long last, Regional assignments are out, and that means it’s time for some analysis! We will have our week one analysis out as soon as possible, but in the meantime, we have crunched some numbers to bring you our initial thoughts on the difficulty of each Regional.
Different strength breakdowns:
As usual, we can break this down in a lot of different ways:
First, we look at the average rank for each regional. By looking at the average TPR rank, we get a sense of the overall strength. Ranks are the simplest metric for measuring the strength of teams, a lower average rank indicates better teams on the whole. The first column on our chart represents the average rank of the teams at each Regional.
It’s also important to look at how strong the top teams at a tournament are because these are the teams you have to displace if you want to get a bid. Thus, the second column looks at the average rank of the top “n” teams at each Regional, where “n” is the number of bids allocated to that Regional. The important thing to note here is that we don’t know exactly how many bids each Regional is getting this year. We are basing our calculation of “n” on the formula AMTA lays out in their rules, but as teams drop and get added the number of bids will change and AMTA often changes their bid allocation formula at the midyear meeting.
TPR points look at the raw data that AMTA uses to assign ranks (sometimes there are large jumps in rank so this can be a little more accurate). For TPR, we can look at the sum of the TPR for all of the teams there which gives us a sense of the total power at the tournament. Our third column looks at the total number of TPR points present in a region.
We can also look at the average number of TPR points for a team in a region to take into account the fact that different tournaments have different numbers of teams. This looks at how strong the average team at the tournament is. Our fourth column looks at the average TPR points of a team in the region.
Once again, we can use TPR to look at just the average of the top teams. This tells us how much power you have to displace to get a bid. The fifth column looks at the average number of TPR points held by the top “n” teams in a region, where “n” is the number of bids allocated to that Regional.
And finally, we can look at the number of teams from various difficulty levels at each tournament. Our chart shows the number of ranked teams (column G), Top 200 teams (column H), the number of Top 100 teams (column I),the number of Top 50 ( column J) teams, and the number of teams that competed at the 2022 NCT(column K) in each region
We have taken the liberty of color coding our chart so that it’s easy to see where each region stands relative to the rest of the field in each category. Red regions are harder than others in that category. Green regions are easier than others. The darker the red, the harder, the darker the green, the easier.
https://docs.google.com/spreadsheets/d/1SdHmey0OCu5oUWWhyx4zugkhvMepxGGHupmkZ8JilWc/edit?usp=sharing
**A note on C, D, and E teams: since C+ teams can’t get a TPR, but some of them are quite good, we always try and give them a ghost TPR. We have updated our formula this year to try and take into account the way different programs approach their C teams. Some programs have a huge drop off after their B teams because they know their C+ teams can’t advance. Some maintain consistently strong C teams year after year. In an attempt to account for teams like the University of Florida, who consistently bids 5 teams from Regionals, while also accounting for programs with massive drop offs, this year we have assigned C+ teams TPR based on how often the program gets enough bids at Regionals to accommodate the team (i.e., three bids for a C team, four for a D team, etc.).
***A Note on Unranked teams: For each unranked team (A/B teams with no TPR or C+ teams that have not qualified for the last three years), we assigned a TPR score of 0 and a rank of 280.
The MAIMD Official Ranking:
And now, averaging the results from our different ranking methods, we have the official Mock Analysis Regional rankings from most to least difficult:
Tracking Parity:
Last year, we worried that coming out of a year of non-geographical tournaments produced weirdness. More Midwestern teams made it to the ORCS level and fewer coastal teams did at the online regionals. But the coasts far outperformed their usual averages in terms of getting teams to NCT. That skewed TPR so that there were more Coastal teams with very top TPR’s and more Midwestern teams with medium TPR’s than in the pre-COVID landscape. The result was an original regionals assignment (before things went online) with significantly less parity in TPR, not because the assignment committee did anything particularly weird last year, but because TPR was showing the disparities in strength between regions more than usual. In other words, normally some regions have artificially inflated TPR or artificially deflated TPR as a result of the difficulty of the fields in which their teams usually play. This creates an artificial parity. But last year, TPR reflected what happens when geography doesn’t matter. This non-artificial TPR highlighted just how little balance our normal regionals really have.
But then last year, we managed to have a hybrid season with an even weirder effect on TPR. Regionals were online, which generally results in more Midwest teams getting ORCS bids. But ORCS was back in person, meaning that the coasts didn’t get the online boost in National Bids. What that means is that we should expect the teams at the bottom of the TPR list to have fairly accurate TPRs after two years of non-geographic regionals. But the teams at the top of the TPR list should have started to drift back to the normal artificiality as a result of geographically constrained ORCS with artificially inflated TPRs in the middle of the country and artificially depressed TPRs on the coasts.
This has resulted in some truly bizarre effects on parity. To AMTA’s credit, after the massive drop off in parity last year, they clearly redid a lot of the traditional assignments in ways that shook things up. We have a lot of teams traveling to new places for Regionals and an increase in top teams flying across the country to increase parity in the weaker regions–for instance, the Jumbos of Tufts flying out to Oklahoma. As a result, when we look at the overall averages in the Regionals, they look like they have returned to 2019 levels of parity, despite the shift to a less geographically distorted TPR. This means that AMTA achieved TPR balance without being able to rely on the geographic inflation and deflation as a crutch that naturally creates some balance in the regions. For that, the team and feeder committee deserves a pat on the back.
The downside to this is they had a pretty clear strategy for creating balanced Regionals. There are a couple Regionals in weak areas of the country where all of the power is coming from a few teams that have been flown in. For example, in Norman, there are only five teams that have TPRs at all (as compared to many Regionals that have 10+ ranked teams). But those two ranked teams are Tufts and Rhodes. Those Regionals have astronomical averages when we look at our top team metrics (significantly harder than either of the last couple years). This results in a huge loss of parity year over year in terms of the top team metrics.
Practically speaking, this means that teams can expect a very different experience based on where they get sent. Getting sent to a tournament with a smoother distribution of power (i.e., a couple fairly strong teams, a lot of low ranked teams, and some unranked teams) is likely to mean that a mid-level ranked team has a good shot of winning in each round, but also a good shot of losing in each round. So they must be on their game the whole tournament, but they won’t face any teams that will just smash them to pieces. Getting sent to a tournament with a skewed distribution of power (one or two insanely high ranked teams, only a couple other ranked teams, and a lot of unranked teams) means that a mid-level ranked team should have a fairly easy time with most of their rounds, but also face one or two rounds that will be near impossible to win. It also opens up a greater possibility of someone sneaking through, having escaped the few elite teams at the tournament. Depending on what kind of team you are, one type of field could be significantly more favorable.
Final Thoughts:
As always, we’d like to note that while these statistics are helpful in understanding the makeup of each Regional, they aren’t definitive. Teams with high TPRs and competitive histories underperform all the time, and teams without particularly competitive histories often surprise us with a breakout season. At the end of the day, we do these writeups to help competitors connect with and understand the AMTA community–not to make anyone feel like they don’t stand a chance because they’ve been assigned the hardest Regional. Take these numbers as an opportunity to understand the field you’re up against, but don’t use them to count yourself out. No TPR or field balance–be it competitive or not—means a thing once you actually step into the well. Good luck to everyone this season!
Different strength breakdowns:
As usual, we can break this down in a lot of different ways:
First, we look at the average rank for each regional. By looking at the average TPR rank, we get a sense of the overall strength. Ranks are the simplest metric for measuring the strength of teams, a lower average rank indicates better teams on the whole. The first column on our chart represents the average rank of the teams at each Regional.
It’s also important to look at how strong the top teams at a tournament are because these are the teams you have to displace if you want to get a bid. Thus, the second column looks at the average rank of the top “n” teams at each Regional, where “n” is the number of bids allocated to that Regional. The important thing to note here is that we don’t know exactly how many bids each Regional is getting this year. We are basing our calculation of “n” on the formula AMTA lays out in their rules, but as teams drop and get added the number of bids will change and AMTA often changes their bid allocation formula at the midyear meeting.
TPR points look at the raw data that AMTA uses to assign ranks (sometimes there are large jumps in rank so this can be a little more accurate). For TPR, we can look at the sum of the TPR for all of the teams there which gives us a sense of the total power at the tournament. Our third column looks at the total number of TPR points present in a region.
We can also look at the average number of TPR points for a team in a region to take into account the fact that different tournaments have different numbers of teams. This looks at how strong the average team at the tournament is. Our fourth column looks at the average TPR points of a team in the region.
Once again, we can use TPR to look at just the average of the top teams. This tells us how much power you have to displace to get a bid. The fifth column looks at the average number of TPR points held by the top “n” teams in a region, where “n” is the number of bids allocated to that Regional.
And finally, we can look at the number of teams from various difficulty levels at each tournament. Our chart shows the number of ranked teams (column G), Top 200 teams (column H), the number of Top 100 teams (column I),the number of Top 50 ( column J) teams, and the number of teams that competed at the 2022 NCT(column K) in each region
We have taken the liberty of color coding our chart so that it’s easy to see where each region stands relative to the rest of the field in each category. Red regions are harder than others in that category. Green regions are easier than others. The darker the red, the harder, the darker the green, the easier.
https://docs.google.com/spreadsheets/d/1SdHmey0OCu5oUWWhyx4zugkhvMepxGGHupmkZ8JilWc/edit?usp=sharing
**A note on C, D, and E teams: since C+ teams can’t get a TPR, but some of them are quite good, we always try and give them a ghost TPR. We have updated our formula this year to try and take into account the way different programs approach their C teams. Some programs have a huge drop off after their B teams because they know their C+ teams can’t advance. Some maintain consistently strong C teams year after year. In an attempt to account for teams like the University of Florida, who consistently bids 5 teams from Regionals, while also accounting for programs with massive drop offs, this year we have assigned C+ teams TPR based on how often the program gets enough bids at Regionals to accommodate the team (i.e., three bids for a C team, four for a D team, etc.).
Most Recent Year With Enough Bids | 2022 | 2021 | 2020 | None of the Last Three Years |
C team | (B TPR)/2 | (B TPR)/4 | (B TPR)/8 | 0 |
D Team | (B TPR)/4 | (B TPR)/8 | (B TPR)/16 | 0 |
E Team | (B TPR)/8 | (B TPR)/16 | (B TPR)/32 | 0 |
***A Note on Unranked teams: For each unranked team (A/B teams with no TPR or C+ teams that have not qualified for the last three years), we assigned a TPR score of 0 and a rank of 280.
The MAIMD Official Ranking:
And now, averaging the results from our different ranking methods, we have the official Mock Analysis Regional rankings from most to least difficult:
- Dayton
- Chapel Hill
- Wheaton
- Fresno
- State College
- Williamsburg
- Columbia
- Buffalo
- East Lansing
- Norman
- Spartanburg
- Columbus
- Claremont
- Princeton
- Springfield
- Tempe
- St. Louis
- Washington
- Owings Mills
- Tallahassee
- Madison
- Des Moines
- Lawrence
- New London
- Providence
- Seattle
- Chestnut Hill
- Evanston
- Waco
- Jackson
- Houston
- Colorado Springs
Tracking Parity:
Last year, we worried that coming out of a year of non-geographical tournaments produced weirdness. More Midwestern teams made it to the ORCS level and fewer coastal teams did at the online regionals. But the coasts far outperformed their usual averages in terms of getting teams to NCT. That skewed TPR so that there were more Coastal teams with very top TPR’s and more Midwestern teams with medium TPR’s than in the pre-COVID landscape. The result was an original regionals assignment (before things went online) with significantly less parity in TPR, not because the assignment committee did anything particularly weird last year, but because TPR was showing the disparities in strength between regions more than usual. In other words, normally some regions have artificially inflated TPR or artificially deflated TPR as a result of the difficulty of the fields in which their teams usually play. This creates an artificial parity. But last year, TPR reflected what happens when geography doesn’t matter. This non-artificial TPR highlighted just how little balance our normal regionals really have.
But then last year, we managed to have a hybrid season with an even weirder effect on TPR. Regionals were online, which generally results in more Midwest teams getting ORCS bids. But ORCS was back in person, meaning that the coasts didn’t get the online boost in National Bids. What that means is that we should expect the teams at the bottom of the TPR list to have fairly accurate TPRs after two years of non-geographic regionals. But the teams at the top of the TPR list should have started to drift back to the normal artificiality as a result of geographically constrained ORCS with artificially inflated TPRs in the middle of the country and artificially depressed TPRs on the coasts.
This has resulted in some truly bizarre effects on parity. To AMTA’s credit, after the massive drop off in parity last year, they clearly redid a lot of the traditional assignments in ways that shook things up. We have a lot of teams traveling to new places for Regionals and an increase in top teams flying across the country to increase parity in the weaker regions–for instance, the Jumbos of Tufts flying out to Oklahoma. As a result, when we look at the overall averages in the Regionals, they look like they have returned to 2019 levels of parity, despite the shift to a less geographically distorted TPR. This means that AMTA achieved TPR balance without being able to rely on the geographic inflation and deflation as a crutch that naturally creates some balance in the regions. For that, the team and feeder committee deserves a pat on the back.
The downside to this is they had a pretty clear strategy for creating balanced Regionals. There are a couple Regionals in weak areas of the country where all of the power is coming from a few teams that have been flown in. For example, in Norman, there are only five teams that have TPRs at all (as compared to many Regionals that have 10+ ranked teams). But those two ranked teams are Tufts and Rhodes. Those Regionals have astronomical averages when we look at our top team metrics (significantly harder than either of the last couple years). This results in a huge loss of parity year over year in terms of the top team metrics.
Practically speaking, this means that teams can expect a very different experience based on where they get sent. Getting sent to a tournament with a smoother distribution of power (i.e., a couple fairly strong teams, a lot of low ranked teams, and some unranked teams) is likely to mean that a mid-level ranked team has a good shot of winning in each round, but also a good shot of losing in each round. So they must be on their game the whole tournament, but they won’t face any teams that will just smash them to pieces. Getting sent to a tournament with a skewed distribution of power (one or two insanely high ranked teams, only a couple other ranked teams, and a lot of unranked teams) means that a mid-level ranked team should have a fairly easy time with most of their rounds, but also face one or two rounds that will be near impossible to win. It also opens up a greater possibility of someone sneaking through, having escaped the few elite teams at the tournament. Depending on what kind of team you are, one type of field could be significantly more favorable.
Final Thoughts:
As always, we’d like to note that while these statistics are helpful in understanding the makeup of each Regional, they aren’t definitive. Teams with high TPRs and competitive histories underperform all the time, and teams without particularly competitive histories often surprise us with a breakout season. At the end of the day, we do these writeups to help competitors connect with and understand the AMTA community–not to make anyone feel like they don’t stand a chance because they’ve been assigned the hardest Regional. Take these numbers as an opportunity to understand the field you’re up against, but don’t use them to count yourself out. No TPR or field balance–be it competitive or not—means a thing once you actually step into the well. Good luck to everyone this season!
- Dinglebingus
- Posts : 14
Reputation : 19
Join date : 2022-03-10
Re: 2023 Regionals By The Numbers
Mon Nov 28, 2022 2:21 pm
I've been waiting for this since regionals got announced! However, your methodology for the overall rankings appears to be biased toward regionals with more teams, with few visible controls from what I can see. From what I can see, this results in underrating especially the Lawrence regional. The only rankings that account for the size of regional are the average rank and top teams rank (admittedly, the top teams rank makes Lawrence seem quite easy). However, I still think the top teams rank gives a misleading indication of difficulty because very good teams skew it more. I think a ranking of the 3 teams to either side of the regional cutoff is a much better indicator (example: if 4 teams qualify, teams #2-#7), since that is actually who regionals competitors will be competing with.
Finally, I'd like to see a preliminary measurement of ORCs strength based on each feeder regional.
Finally, I'd like to see a preliminary measurement of ORCs strength based on each feeder regional.
- Sir Mocksalot
- Posts : 26
Reputation : 12
Join date : 2019-12-16
Re: 2023 Regionals By The Numbers
Mon Nov 28, 2022 3:57 pm
Okay I thought this when I first looked at it, but glad to see some statistical confirmation. Dayton is absurd right now, need some changes there. A swap or two with Columbus could go a long way.
- MockAnalysisIsMyDrug
- Posts : 219
Reputation : 247
Join date : 2019-11-19
Re: 2023 Regionals By The Numbers
Fri Dec 02, 2022 9:48 am
Dinglebingus wrote:I've been waiting for this since regionals got announced! However, your methodology for the overall rankings appears to be biased toward regionals with more teams, with few visible controls from what I can see. From what I can see, this results in underrating especially the Lawrence regional. The only rankings that account for the size of regional are the average rank and top teams rank (admittedly, the top teams rank makes Lawrence seem quite easy). However, I still think the top teams rank gives a misleading indication of difficulty because very good teams skew it more. I think a ranking of the 3 teams to either side of the regional cutoff is a much better indicator (example: if 4 teams qualify, teams #2-#7), since that is actually who regionals competitors will be competing with.
Finally, I'd like to see a preliminary measurement of ORCs strength based on each feeder regional.
Thanks for the feedback on our stats!
Re: Bias towards big regionals. The only stat we rank on that should have any bias towards big regionals is the Total TPR. When we do our final ranking of regionals, we only consider Average Rank, Top Teams Rank, Average TPR, Total TPR, and Top teams TPR. We do put the number of teams in various categories on our chart so people can see it (because we think that’s helpful), but those counts don’t factor into our overall ranking. So most of what’s going into our overall ranking is fully controlled for regional size since the component rankings are average based metrics.
The reason we continue to have Total TPR in there (despite it rating larger regionals harder in general) is actually to more accurately represent the fact that Average TPR/Average Rank tends to overlook the difficulty of being in a region with a lot of strong teams (even if there are also a lot of weak teams). Take two regionals each of which has ten strong teams, and then the rest are unranked. One is a twenty-team regional. One is a thirty-team regional. In some sense, for the teams that are going to bid, these regionals are equivalent because their do-or-die rounds will probably be against those ten teams. But any averaging metric would rate the large regional as way easier than the small regional because the added weak teams will water down the strengths (even if, realistically, you aren’t super likely to hit them). We still think averaging is an important metric because some of your rounds will be against those teams. But we also think having a metric that just looks at the presence of strength without the watering down is helpful.
Re: Top teams. We could in theory expand our definition of top teams, but that tends to be less precise. How big of a range we should do around the cutoff mark is going to vary wildly with what region we are looking at if we want to accurately measure the teams on the brink. As we have discovered doing our normal write-ups, some regionals have a ginormous bubble that basically takes up half the regional and some have a clear cut-off within a few teams of the bid threshold. So, yes, we could have our top teams metric look at the teams just below the bid cut-off, but any threshold we set for what “just below” means would likely be decidedly arbitrary, and we didn’t want to do that.
As for the concern that it is skewed by the presence of really high-powered teams at the regional…that’s kind of the point. The idea is to look at who you will be hitting in the really top rounds.
Re Lawrence: Lawrence is a regional that is hard on some metrics and not on others, and as a result, we stand by our ranking placing it pretty in the middle. It’s got a wider bubble than many regionals which means that teams are likely to have fewer really easy rounds than they would in a more stratified regional but on the other hand, there just isn’t a top team that’s likely to smash everyone (in fact there are no team that made Nationals last year). In other words, for most bubble teams, you have to go into that region prepared to play consistently well, but you don’t have to worry about just getting steamrolled by one team, automatically dropping two ballots, and having to be perfect everywhere else to make up for it. Compare that to the region right above it in our rankings, Des Moines. Des Moines has a lower average rank/TPR because its a much bigger region and has a lot of weak teams at the bottom but anyone on the bubble has to be terrified of the prospect of hitting a Chicago A who they just don’t have nearly the same shot at beating as they do the top teams in Lawrence. A bubble team going into Des Moines has to be prepared to hit Chicago A, meaning they have to run the table on every other team they could hit. In other words, Des Moines is just as hard as Lawrence, just in different ways. Both of them are far easier than any of the super highly rated regionals, many of which are just stronger in every metric.
Re: ORCS stats. The problem is, right now, the way AMTA has assigned bids to feed would send more than 24 teams to New Rochelle, Santa Monica, and Washington (because those are where a lot of the huge regionals feed to). So we have to assume they are going to either change where some bids feed closer to regionals or move some teams. Until we know what they are planning, we can’t predict what will happen to the ORCS.
Permissions in this forum:
You cannot reply to topics in this forum