March Madness: Ranking the ranking methods

How well do various methods perform in predicting the winners of the NCAA men's and women's basketball tournaments — the annual rites of spring collectively known as “March Madness”?

Interest in the tournaments is immense, as attested to by the many millions of fans who fill out brackets each year in hopes of winning pools informally organized in office-workplaces and gyms. Most employ algorithms of their own devising, which may involve careful scrutiny of a season’s worth of statistics, but more likely are based on nothing more mathematically rigorous than the color schemes of team uniforms or an affinity for a particular school's mascot.

There are no fewer than 147,573,952,589,676,412,928 different ways to fill out a 68-team March Madness bracket. However, actual tournament outcomes in recent years suggest that the model which has acquitted itself the best overall is the “Logistic Regression/Markov Chain” (LRMC).

Its key component is a tool developed more than a century ago by A. A. Markov, a Russian mathematician. Markov invented his “chains” to analyze apparently random sequences of vowels and consonants in an epic poem. They have since been used in a great many other ways, however.

Throughout the college basketball season, for example, fans with a predilection for advanced number-crunching bring LRMC to bear on the basic scoreboard statistics for matchups involving NCAA Division I teams. Included are primary data such as wins-and-losses, margin-of-victory and the court on which each game is played.

The model considers strength-of-schedule and evaluates teams not just by looking at whether a matchup ended in a victory or a defeat for a particular team, but also the points scored for-and-against, and the quality of the opponent.

On the second Sunday of March each year — “Selection Sunday” — the tournament field is announced and the 68 teams invited to compete are placed within brackets for seeding purposes. In general, higher seeding is preferred, since it results in being matched up with lower-seeded teams in the early rounds of the tournament.

Baback “Bobby” Vaziri, Ph.D., is an associate professor of Business Analytics and an expert on systems of rating and ranking as they apply not only to sports but other pursuits as well. He says that Markov-based approaches are “very powerful but also very sensitive,” and that, precisely because they are so powerful, “a small perturbation in your data can give you very skewed and inaccurate results.”

He cites the example of the chaos wrought in the model by the historic upset in the opening round of the 2018 men's tournament, when a top-ranked and top-seeded University of Virginia team lost to the 16th-seeded team from the University of Maryland–Baltimore County.

“What method could have predicted that?” Vaziri exclaims. 

He says that, “When we start talking predictively, we connect the dots backward and hope we can then forecast where we’re going in the future.”

That's how models are built, he says. Given the seemingly random way in which events unfold, however, “it can be really hard to do.”

Markov chains can be characterized, says Vaziri, with a single word — “voting.” Every matchup between two teams creates a circumstance in which the loser casts a “vote” for the winner.

In basketball, there are many statistical categories in which competing teams may cast votes for each other. Perhaps the simplest uses wins and losses. A losing team votes for each team that beats it.

A more advanced model uses game scores. In this case, the losing team casts a number of votes equal to its margin of defeat. An even more advanced model lets each team cast votes equal to the number of points it gave up. The number of potential modeling tweaks is virtually limitless.

In the end, the team receiving the greatest number of votes from high-quality teams is awarded the highest ranking. The idea is actually an adaptation of the famed PageRank algorithm used by Google to rank webpages.

In general, the advantages of a Markov-type approach, says Vaziri, are that it 1) takes the strength of the opponent into account, and 2) that it takes the strength of the opponents’ opponents into account.

Thus, he says, “it creates a web of knowledge that takes into account not only how you have performed against your opponents, but also how well your opponents have fared against their opponents.”

In a series of published papers, Vaziri has proposed an extension of the classic Markov method that corrects the “mathematical pathologies” to which it becomes prone in a hyper-sensitive state.

It is more fair and comprehensive than rival methods of ranking, he says, in that it 1) rewards a team more for beating a strong opponent than for beating a weaker one, 2) always provides a team with an incentive to win, and 3) gives no weight to the sequence in which games occur within the season schedule.