Jan 5
Beta - Do It Differently
I’m catching up with Holiday blogging, so you’re going to see some stuff about the end of the year this weekend. I’m hoping to get everything posted out so that I can pick up where I left off on Monday. So yay!
I wanted to point out a post by Moorgard from the last week of December about the uselessness of MMOG Beta tests. From his post:
It is a prevalent attitude among today’s MMO players that beta is a time to evaluate a game and decide whether it’s worth buying when it officially launches. Others see beta as a free trial period during which they can run through the game, learn its secrets and shortcuts, and get a nice head start on other players who will have to learn all the basics at launch. Both these attitudes are exactly wrong, and are part of the reason why it’s so hard to find good beta testers these days. Now granted, part of this is the industry’s fault.
I want to be ranty and say ‘It’s completely the industry’s fault, and game companies suck’, but I know better. In fact, while I agree that marketing has muddied the waters quite a bit with giveaways and such, I very much think player expectations are what have screwed up Beta tests. I am constantly amazed by the attitude of some gamers. To hear them talk, you’d think the very idea of a company making a game without consulting them is the height of stupidity. This is kind of a sticking point with me, and I think reflects on the psyche of American citizens in general. This attitude of entitlement is something I can go on about at length - ask me next time we’re in a bar together.
The point here is that the Beta test has to be about one thing - it’s either an event for the game’s makers, or it’s an event for the fans. If it’s an event for the game makers, the purpose of the Beta is to refine the game. The people ‘in’ the game should be playing it, yeah - it has to be fun. But there should be a purpose to the fun. There should be a process - individual elements tested, iteration on portions of the game, actual testing. If it’s for the fans, there should be events! With the small player population, you can do things that you can’t do once the game is out. You can create hype for the game, establish lore, make players want to play the game with their friends. Bump people to the higher levels, make people want that endgame content.
As someone who has been in a number of Betas, I always try to make sure I report a bug after every session. I’ve talked to a number of people that seem to think that Betas should be free gameplay time … if’n you want that, man, go become a game journo.
1 Comment so far
Leave a comment





You’re been too hard on the people who just want free gameplay. For the specific case of MMOGs, bulk players are valuable. You test scalability. You see get a better sense of what real players are prone to do in your game (you do have automated logging and tracking, right?). You should have automatic crash dump reporting systems on both the servers and clients, so you’ll get automated bug reports from them. Lots of good stuff.
Even if they’re just looking for free gameplay, well, what’s the harm in that? That’s what the developer offered. Sure, I have no sympathy for someone who feels entitled to free gameplay, but I suspect you’re inflating the actual number of people who feel that way. Where are you seeing these people? Web forums? You know that they’re a self-selecting, unrepresentative sample of humanity. Web forums for MMOG doubly so. Even the most selfish beta-player knows that the beta will end eventually and understands it to be a special case for which they should be thankful.
If these beta players who don’t report bugs are really so horrible, it wouldn’t be hard to filter them out. They play at the mercy of the developer; kick the non-useful ones and invite more people in to replace them. I suspect very simple rules would be effective at filtering out 90% of them. “No bug reports in X hours of play? Kick.” “All bug reports are just gibberish designed to avoid previous rule? Kick.” “Bug reports match word for word bug report from 10 other people? Probably trying to avoid first rule. Kick.” If they’re feeling really clever, insert intentional “bugs” and see who reports them.
I also don’t think it’s realistic to make a better for just debugging or just for hype. A typical MMOG needs to do both. Assuming infinite funding to develop the game, yes, it would be ideal to do beta testing first, then when the game is ready open it up to a limited hype-period. But that’s not realistic for companies with finite budgets. The beta period needs to do double duty.
As for creating special beta content; that’s just silly (although I know some developers do it anyway). The more “special” your beta period is, the less it reflects actual gameplay. It makes it less useful for debugging as it will tamper with gameplay. It makes it less useful for hype as it will create unrealistic expectations for the final release.