Getting to the Points with Aaron and Scott

Posted in The Week That Was on September 9, 2011

By Brian David-Marshall

Brian David-Marshall is a New York–based game designer who has been involved with Magic since 1994, when he started organizing tournaments and ran a Manhattan game store. Since then, he has been a judge, a player, and one of the longest-tenured columnists on, as he enters his second decade writing for the site. He is also the Pro Tour Historian and one of the commentators for the Pro Tour.

As explained in this week's announcement by Mike Turian, the long-running Elo rating system that has been in place since the beginning is being replaced by a new system called Planeswalker Points. It is going to completely change the way your rating works and who gets invited to Pro Tours, and it will make Friday Night Magic more meaningful than it has ever been since it was introduced way back in 2000.

Here to give us some more insight about the program are Magic R&D Director Aaron Forsythe and Magic Organized Play Program Manager Scott Larabee.

Scott Larabee and Aaron Forsythe

BDM: Planeswalker Points are replacing the Elo rating system. Why change something that has been a part of the Magic experience for such a long time?

Scott Elo has a lot weaknesses in that it sometimes encourages players not to play. You hear these stories all the time. Players finally get over 1800 and they are going to get one bye at a Grand Prix that is coming up in their hometown four months later. And they just stop playing. They don't play in Friday Night Magic—they don't play in anything. They don't want to do anything to jeopardize their rating. One of the things that the new Planeswalker Points system does is that it is an accumulation system. You can't lose points for playing matches—you get three points for a win, one point for a draw, and zero points for a loss. But if you lose you don't go negative like you do in Elo. The one-sentence description of Planeswalker Points is: "Playing is good. Winning is even better."

Aaron Elo did a good job of legitimizing Magic as an intellectual sport when the game was in its infancy, because you could see that certain players were better than others. If you went on a winning streak, it quantified that. You knew who the best players in your city were, who the best players in the world were. We have been paying a lot of attention to what people in the gaming world have been doing in recent years, and across the board—whether it is MMOs or social games—the way you keep people invested is through a constant stream of positive reinforcement. And Elo was very bad at that—it didn't deliver that.

Elo is really good for matching players of a certain level—and a lot of video games will use it behind the scenes for auto-matching, which it is great for—but for the things we need to use rating for, like inviting people, it is not as good.

We don't use Elo for matching. We randomly pair people at the start of a tournament and then we match you based on your record for the day. So we don't need it for that. I think that we don't need to prove that Magic is a skill game anymore. We have fifteen years of data that proves that. I don't need a formula to prove that the better players win more often. We do need a way to say that it is okay that you lost and that you are making progress. We want you to come back. We don't want people saying the best way to qualify for events is to not play. That is just ridiculous and defeats the purpose of the whole system.

BDM: Can you explain what "Playing is good" means?

Built into the system are participation points. If you come to Friday Night Magic and you play but you go 0-5, you still get points because of the participation points built in. And to differentiate the different types of events there is a multiplier system built in. The more prestigious the event, the higher the multiplier is. All of these things will address the problems that Elo has and is a more familiar rating system to those who play video games.

For newer players the experience was often like, "Welcome. You have lost. You have spent all your money, and then when you go home and look it up on the Internet we will prove to you that you are terrible. We have quantified that you are worse than a player who has never played at all."

BDM: What about the "winning is better" part?

The first thing that we have done is that when we launch, there are four different types of point total. First is Lifetime, which counts everything. It counts casual events. It counts competitive events. Everything. The only thing Lifetime is used for is your level. Then we have Competitive, which covers anything that is not Casual. You also have Friday Night Magic, which only counts your Friday Night Magic points. Then there is Professional, which counts only Grand Prix and Pro Tour. These categories overlap, so the points you get in a Friday Night Magic tournament will feed into your Lifetime, Competitive, and Friday Night Magic totals.

We are going to use these different totals for different things. Lifetime is used just for your level. This is something people can look at to gauge everything they have ever played and where they stack up. Competitive is going to be used for Pro Tour invitations, Nationals invitations, and byes at Grand Prix.

We are going to redefine some terminology here. People historically have thought of two kinds of events: you have events that are either sanctioned or unsanctioned. That is going to be changing. Everything is going to be sanctioned. Everything that was "casual non-rated" is now just Casual. You still get Lifetime points for it, so it is not "non-rated." And everything you used to know as sanctioned is now Competitive.

Causal Points and Competitive Points work differently. On the Casual end of things, you get 1 point for participating in a Casual event, and there are no matches recorded. Even if you run a Casual event that has matches, we don't care. You show up and you get credit for playing.

For all the Competitive stuff, matches matter. Friday Night Magic is going to be used for the Friday Night Magic Championship, which we are going to run in 2012.

BDM: This sounds a lot like making the Top 8 of a multiple Pro Tour Qualifiers could have the same result as actually winning one.

Definitely. This system rewards you the more you play. If you are one of those players who makes the Top 8 a lot—if you always get close—it is going to add up. Before it didn't add up; it was just yes or no. Those are the people we want to reward. We don't want to reward the player who has been sitting on an invite for three years waiting for a Pro Tour to come within a hundred miles so it is convenient. Sorry. I know people do that but those are not the players we should be rewarding. Those are just not the people to we want to say, "Good job, why don't you come to the Pro Tour?" to.

Each season has a start date and an end date. Sometime shortly after the end date we are going to pull the Top X of whatever we are inviting to. For Pro Tour Honolulu the season ends at the end of the year—basically—and we are going to invite the Top 10 people from Europe, the Top 10 people from North America, then the Top 5 each from Latin America, Asia Pacific, and Japan, and then we are going to take the next 65 players overall (who didn't already get an invite that way) as well. And they are all getting plane tickets as well.

BDM: So in addition to PTQ winners getting plane tickets the rating invites are going to get flown there as well?

It is a huge change. The ratings people are going to get tickets as well.

BDM: Worlds in San Francisco will use Elo-based ratings, correct?

Worlds in San Francisco will use the Elo system to invite, and it will be the last event that uses that system. After that you might as well start playing again. The people we are inviting next year are not the people who have been sitting on their rating for the past two years and slowly incrementing it up and then getting it to a point where they think it will invite them to a Pro Tour so they stop playing for six months.

BDM: It really seems to keep coming back to incentivizing people to play instead of giving them reasons not to play. There have certainly been some independent articles on the subject that have popped up over the last year.

The more stories I saw about people sitting on their rating—it just defeats the whole purpose. If Elo was just a detached measure of your skill that would be one thing but the fact that there are invites and byes on the line means that gaming the rating becomes the whole point of playing at all. That just defeats the purpose. Rating is supposed to be a measure of how good you are—you should not be able to shape rating to be something other than what it is. Alex West and Brian Kibler have written some articles about this and I have had to bite my knuckles not to shout, "We get it, and we're working on it! You will get to play soon!"

It may look like we are a little late to the party but we have been working on it for years with fifteen years of technical debt to pay.

BDM: We are not starting at square one with our ratings? All my past matches will count? I don't get the time to play much anymore.

This thing would never have gotten approved if we all started at square one. One of the things that makes Magic great is that everything from fifteen years ago still works. The cards still work and we wanted to make sure that the activities that you participated in back then still count too. Right now it probably gets all washed away by Elo with the fluctuations up and down, but if you have been playing for a long time, with Planeswalker Points it is going to show. You are going to be at a much higher level than someone who started a couple of years ago.

BDM: What do I get from all the different types of ratings? Is my Level 41 Lifetime status going to get me invited to a PT?

The Lifetime ratings aren't used for invites or byes. You can look at them and see where you stand. The Lifetime is always there. It never goes away, it never resets to zero, it just keeps going. The other ratings all reset at different times. The Competitive is broken up into three seasons each year, and when a season starts you go to zero. When the next season starts you go back to zero. At the end of each season we look and see who the people getting invited to a Pro Tour are, who is getting byes, and, if it is a Nationals-relevant season, who is getting invited to Nationals.

The FNM season is a length of time that starts with everyone at zero, and at the end we will invite people to the FNM Championship. Professional is going to be used to invite people to Worlds. At the start of the year everyone is at zero, and at the end of the year we will invite people to Worlds. And then it will start over at zero for the next year.

BDM: I think one of the biggest hurdles to get over in the Tuesday announcement is that Pro Tour invites are being decoupled from Grand Prix. Can you explain why that happened?

If you finish in the Top 16 of a Grand Prix, you are not automatically getting an invite to a Pro Tour anymore. But... Grand Prix are the events that anyone can play in that offer the most Planeswalker Points. Anybody can go play, they are very large—the bonus participation points scale up with the number of players in the event—and they have an 8x multiplier on points. They do not have as high a multiplier as the Pro Tour or Worlds but eight times the participation bonus is a lot of points before you've even won any matches. The thinking is that instead of granting those invites directly, we are going to let our new rating system grant those invites in different ways.

I am sure that there are going to be people disappointed about that, but we have three Pro Tours a year and only so many people that we can have in those Pro Tours without them becoming huge and less fun. We have been trying to increase the number of Grand Prix for years and we have been talking about how to do it without Pro Tours becoming four-day, 800-person events. Those would not have been fun for anyone. We decided that the way we could increase the number of Grand Prix and bring the cool Grand Prix experience to a lot of people was to decouple them from the invites to Pro Tour and instead couple the rating invite to the Pro Tour. And that is what we have done. There will be more opportunities for more people to play in more GPs a shorter distance away from where they live.

BDM: What about byes for Grand Prix? How will those be determined?

Byes are going to work kind of like invitations. You are going to get invites to an event at the end of a season. With byes, at the end of the season we are going to take the top 300 people and give them a three-round bye to every GP they play in that following season. The top 2,000 get a two-round bye, and the top 15,000 get a one-round bye.

BDM: What do you say to people who are upset about the departure from the Elo system that people have accepted as a measure of skill for the game?

I have been on the other side and used Elo to my advantage. I have been on top of the mountain and gotten ratings invites. I know there are people out there who like it, but we have done a lot to grow the game, and we just want to make Organized Play a lot more user friendly and a lot more positive. We want to give people a growth track and a way to measure what you have been doing and keep you coming back for more.

There is a defensiveness and a meanness when good players feel like they are going to lose to bad players. The biggest threat to a good player's rating is being beaten by a bad one who randomly has Circle of Protection: Red main deck. You go on tilt against this person who is going to wreck your rating. So I hope the new change softens the culture a little bit—we had to stop showing Elo rating on MTGO because people were just trolling each other with it.

BDM: How would you have felt about this when you were on the other side of the fence as a player going to PTQs and eventually Nationals and the Pro Tour?

I would love it. I am one of those guys who's in 110%, and it is nice to see that get you somewhere. I am sure that the later Pro Tour version of me would understand why this is better for everyone. The fact that I would have to change my behavior would be significant, but inevitably I would be happier to know that I am free to play. I guess that is what makes me happiest about this; as a PTQ or PT player I don't feel like my status as a pro is at stake every match I play.

I can try some new decks. Nothing is on the line really. There is no way you would try some crazy Mono-White Infect deck under the current system. You could lose, lose, lose and have to win like fifty matches to get back on the Pro Tour. Now you can just give something a whirl—try something crazy. Go play with some folks you have never played with before. It is not going to hurt you. It only helps you.

BDM: You have talked about wanting to change this for a while now. What took so long?

It has taken far longer than I could have ever anticipated. To me it felt like we could just go in there and change the formula from this to this. As we looked under the hood of our Organized Play software we realized how many things needed updating regardless of these changes. Once we started messing with it we had to fix the whole thing. It was a little naïve of me to think we could just snap our fingers and change the way you get your points, but Mike Turian has done a great job of shepherding this through.

BDM: When you look at this program a year from now, what will be the hallmark of this being a success? If it goes smoothly without any hiccups?

There are going to be kinks all along the road once this is in motion. We do not know what player behavior is going to be, but if someone who never played in Friday Night Magic because of ratings concerns goes and plays Friday Night Magic, it will be a success.

That is the top end. For me it is the number of people who are willing to come back for a second try. There are a lot of people who play in one organized event a year and don't come back. We can't put our finger on the exact thing that makes them not come back but the lack of positive reinforcement has to be a big part of that. We think this will greatly increase the amount of people who come back for time number two. This is a huge thing for getting people engaged in the Magic community.

BDM: If you have one closing thought you want to share...

The website is awesome. The ability to look at your entire history is amazing.

We have converted a lifetime of ups and downs to all ups. We have spent enough time on this system to ensure that it rewards skill but it also rewards other traits like perseverance and dedication and that ultimately this system is more positive for more people while still doing all the things that Elo did.

Latest The Week That Was Articles


January 8, 2016

Five Formats in the New Year by, Brian David-Marshall

Two-Headed Giant | Booster Draft | ModernStandard | Canadian Highlander | Player of the Month The sweet sound of Oath of the Gatewatch packs getting cracked will make its way around th...

Learn More


January 1, 2016

Oath of Nissa by, Brian David-Marshall

Do you remember back when blue got all the fun toys? Now, you might think I am talking about cards like Force of Will or Control Magic, but I am actually thinking a little smaller—a lot s...

Learn More



The Week That Was Archive

Consult the archives for more articles!

See All