The Five Trials of Playtesting

Posted in Beyond the Basics on June 1, 2017

By Gavin Verhey

When Gavin Verhey was eleven, he dreamt of a job making Magic cards—and now as a Magic designer, he's living his dream! Gavin has been writing about Magic since 2005.

Growing up, I played a lot of sports. And like many kids in the '90s who played sports, in summer I'd go to some kind of sports camp.

One particular year, the sport on the docket was basketball. It was a time before I had discovered Magic, a time when Seattle still had a professional basketball team, a time when my height was still recorded on the back of my bedroom door.

I wasn't the best basketball player. My growth spurts were less vertical and more horizontal, and my grasp of basketball strategy mostly came from hours spent on NBA Jam.

But, though I wasn't the best basketball player in the room, I was one of the most determined.

I spent lunch and the time after camp activities ended for the day practicing shooting baskets. I'd stand there, surrounded by an empty court and vacant bleachers, just shooting baskets. Shoot the ball. Run and get the ball. Position myself. Shoot the ball again.

Repeat.

Repeat.

Repeat.

The number of baskets didn't seem to be going up. But I did get a lot of practice shooting, and I felt good about that.

This routine continued, exactly like this, until one particular event made me change my methodology.

There I was, shooting. Retrieving. Shooting again. Nothing but me and complete and utter empty—

"Hey there."

My head twirled around. Unbeknownst to me, my coach was in the room. She had snuck in and been watching from the far end of the bleachers.

It was the ideal setup for every high school sports movie pep talk you've ever seen.

"You know, have you tried holding and shooting the ball like this?" She held my hands, positioning them to the right spot.

I took a step forward. I shot the ball.

It went right. Through. The hoop.

"Practice makes perfect," I said, beaming.

"No," she replied. "Perfect practice makes perfect."

She paused. "Practice poorly, and you'll perform poorly. Practice perfectly, and you'll play just as well."

I've never forgotten that moment.

In Magic, you probably hear the word "playtest" a lot.

Perhaps, "I want to playtest this new deck and see how it does," or "Let's playtest a matchup before the PTQ." Whether you're a Commander aficionado, a draftaholic, or a hardcore Standard player, you've almost certainly wanted to playtest something at some point.

And it's crucial. In a game with so many interactions, so many choices, and so much to consider, you need to learn as much as possible.

But here's the caveat: perfect playtesting makes perfect.

Playtesting just for the sake of playtesting is using your time inefficiently. And Magic is a lot more complex than basketball. Each card choice in your deck is a decision, each matchup a cascading network of options. There's no time to waste.

What are you testing for? What are you actually learning? Why are you even playing these games?

Today, let's talk about the best ways to playtest for Constructed tournaments.

#1—Have Goals

So you have your playtesting team or partner. They're the Luke to your Han, the Posh Spice to your Sporty Spice, the Reid Duke to your Owen Turtenwald (or Huey Jensen.) They're also interested in preparing for this tournament. You sit down, and you start playing.

Well . . . why?

If I walk by, and you're playtesting, and I ask you why you're playtesting, you should have an answer for me.

If you need to imagine me patrolling the hallways of your house and asking this question during game night, then do. But always have an answer to this question.

First, figure out what you're hoping to learn from these games. Then figure out how you're going to obtain that information.

Let's take a pretty common scenario: you have an event coming up, and you want to see how your deck does against a specific other deck. You sit down to play.

What are you hoping to learn here?

Here's a list of just a few reasons, all of which could potentially be good goals to work toward:

  • Which deck is favored in the matchup?
  • Which are the most important cards in the matchup?
  • Which cards aren't pulling their weight?
  • Which sideboard cards will help out the most?
  • When should I be aggressive and when should I be defensive?
  • Which cards do I need to remove or counterspell and which ones are fine to leave on the board?
  • How am I winning the games I win?
  • How am I losing the games I lose?

There are tons of potential reasons, far more than listed here—but you should figure out which ones you want to learn and then playtest accordingly. You can work toward more than one at a time, but you should always be working toward at least one.

For example, if your goal is to identify if it's better to be aggressive or defensive in a matchup, then you should try playing different ways in each game to see which works better. If you want to figure out which cards aren't doing much, you should take down notes whenever a card seems to be underperforming.

Have goals.

#2—Gather Data

It's important you play enough games to use that data to make good decisions. Playing one or two games doesn't always tell the entire story.

Of course, on the other hand, there's only so much time. You want to be efficient with your playtesting time.

So, what's the best thing to do?

I usually recommend starting with a best-of-ten set, alternating who plays first. At the end of that ten-game set, as well as during the games, talk with the person you're playing with about what seems to be working and what isn't.

But here is what you absolutely should not do: see a 7-3 record and write it up as a 70%–30% matchup.

Whether you won or lost a lot in playtesting is vastly less important than how you both felt the matchup was. I've had playtesting sets where I've lost 8 out of 10 games, and both players agreed it was in my favor—and vice versa. Ten games isn't close to a large enough sample size to get perfect matchup result data . . . but it is enough to start reaching conclusions if you pay close attention to how the games are going.

Talk a lot with your playtesting partner or team. Come to conclusions as a group about what seems to be working and how matchups seem to feel. The end result of playtesting isn't about generating raw matchup numbers or filling out a spreadsheet of which matchups are advantaged where; it's about figuring out how to use all of the information you've collected to make the best decisions possible.

#3—Play Post-Sideboard Games

I really can't say this enough: playtest post-sideboarded games.

Tournament matches are best two out of three, so you are going to play at least half of your games using a sideboard . . . and almost certainly more.

There are so many matchups that if you just tested the first game you'd think are slanted toward one deck—but quickly become apparent that after sideboarding the other deck has the advantage.

Let's say you're playing an aggressive white deck against a blue-red control deck. You keep winning Game 1 with few difficulties.

However, after sideboarding, the blue-red deck brings in a plethora of burn spells and full set of Sweltering Suns! Suddenly, you can't stick a creature for more than a turn and your entire game plan falls apart.

Now, if you had played post-sideboarded games, you would know that you needed resilient creatures in your sideboard to fight them off. This can easily be the difference between winning and losing a match.

If you're only going to play a ten-game set as described above, I'd do something like four pre-board and six post to give you the best feel for it. But often, you'll want to complete the ten-game set and then talk about switching to several sideboarding games. And, of course, it has to do with what your goals are: if your goal is to specifically figure out what's key in Game 1, then playing post-sideboard won't help you much!

#4—Play Both Sides, Play Everything

"If you know the enemy and know yourself, you need not fear the result of a hundred battles." Or so says Sun Tzu, a monumental Chinese philosopher.

But it's true: just as important as knowing your own deck (and perhaps even more so) is knowing your opponent's.

Playing just your deck is only half the story. When playtesting a matchup, I like to try out both sides. You can learn a lot in just a couple games spent playing as the opponent about which cards you're scared of, what the turning point is, and what your general strategy is.

This, of course, quickly extends out to everything.

Now, when I say everything, what I really mean is "everything you expect to play against in the tournament." While I'm all for playing awesome new decks, unless you're the one playing it and your goal is to learn how the matchup is, playtesting against them isn't nearly as informative as playing against the decks you actually expect to encounter at the tournament.

In a perfect world, I'd like to have played every deck I expect to play against in a tournament and understand how they operate. Time available is certainly a factor, but at least for the major matchups you expect to play against, I highly recommend at least playing those decks some.

The best player in the room is often not the one who has most mastered a deck—it's the one who has most mastered the format.

#5—Experiment!

Playtesting is just that: testing.

You don't want to be the scientist who takes all their lab results, uses them to make a new formula, and then decides that inexplicably the first time they'll test out their formula is in front of a live audience. If there you're looking for a time to try something out, playtesting is the time for it!

If you're playtesting and want to try out a new card, sideboard in a different way, or take a different strategy in the matchup, talk it over with your playtesting partner and give it a try. Maybe it'll work, maybe it won't, but you can always try it out and see.

Experiments can be time consuming, so if you need to shortcut, you can even try breaking the normal rules of Magic by putting that card into your opening hand a few times. (Or, conversely, seeing how it is if you only ever draw it late in the game.) Remember, it's playtesting: do what you need to modify your games to get to your goals—provided you aren't spoiling the data in some other way. (Or, if for some reason, you're playtesting during a tournament. In which case, please don't cheat.)

Try out new sideboard plans. Try new cards. Add or remove lands. Be crazy. Of course, keep in mind how these may impact your other matchups—if you keep adding new cards to your sideboard, for example, your other matchups could be drastically impacted by what you remove—but it's a great chance to learn.

You never know what playtesting might show you

Your Prize

I don't play basketball anymore. Not competitively, anyway. The Seattle SuperSonics moved away. I grew up.

But every activity in your life leaves a lasting impression in some way—and that desire to always try to make perfect practice lingered on.

Years later, to my surprise, I'd find my basketball coach's advice echoed in a similar way by Sarah Kay, one of my favorite poets. Amidst her musings and metaphors about love from the poem (appropriately titled) "Postcards," Sarah wrote, "Practice does not make perfect. Practice makes permanent."

Well, may your perfect practice make perfection permanent.

Whether you're a tournament veteran or new on the scene, from Friday Night Magic to the Pro Tour, I hope this helped give you some direction when playtesting.

Do you have any comments, thoughts, or questions? Please, feel free to contact me via Twitter, Tumblr, or by sending me an e-mail in English at BeyondBasicsMagic@Gmail.com. It's always great to hear from you!

Enjoy playtesting, and have a great time playing Magic! I'll talk with you again next week.

Gavin
@GavinVerhey
GavInsight

Latest Beyond the Basics Articles

BEYOND THE BASICS

October 26, 2018

Playing One-Drops in Limited by, Gavin Verhey

Creatures that cost one mana are as old as Magic itself. Savannah Lions. Llanowar Elves. Mons's Goblin Raiders. And 25 years later, well . . . some things about Magic are still true. ...

Learn More

BEYOND THE BASICS

October 11, 2018

Do(n't) Play Lands by, Gavin Verhey

One of the very first rules you learn when you start playing Magic is that you can play a land every turn. We learn right away the benefits of playing a land each turn as we move from cas...

Learn More

Articles

Articles

Beyond the Basics Archive

Consult the archives for more articles!

See All