Beta Fatigue

I am a very busy guy lately. Work is rediculously busy, family is busy trying to enjoy the summer, and a lot of games are getting closer to release. Since when was summer beta season? I am in 5 right now.

With little time to play any.

I manage to do a little in each each week but nothing I could dedicate in the past. This problem stems from two separate issues, real life being one and beta fatigue the other.

We seem to be in the middle of the WoW-Dev cycle – games coming up for release started during WoW-Prime, so pitches and design are WoW-Like. Sure, there are a couple that stray from the model but the underly is firmly WoW-Entrenched. It is wholly uninspiring to even play these games as I have already spent years in WoW. The minor nuances do not change the core gameplay. Uninspiring is the one word I will probably repeat a lot.

That’s not to say each aren’t going to have their niche and none of them have redeeming features. None of them will pry cash from my wallet though, either. Then again, I am not the typical MMO player either. I am sure these companies realize they have a large audience out there dying to pay $15 a month for something new.

My wife hates how I watch movies – no matter how bad it is, I can’t NOT watch the whole movie. I have seen some really terrible ones because of this. I sit through the 1.5 edited hours waiting for that what redeeming plot twist, memorable quote, that “thing” that made people put money into the project in the first place. Surely, there has to be one thing to make the hour and a half long torture worthwhile? Typically there is and that leads to a level of satisfaction. My wife already started doing laundry 45 minutes ago, but I am happy to pass the one highlight she missed onto her with enthusiasm and “I told you so”.

I used to care a lot more for the beta process – in the past they felt like an actual mechanism for improvement and testing. Lately it seems as though it is just another ’10 day trial’-must-have-mechanism. Some of these betas have slim to none dev communication. While I always try to be a positive participant in the process judging by the quality of communication from fellow testers – in almost all of them – it is not hard to see why companies don’t take their own beta processes more seriously. I know I would be more inspired to test better if the devs actually had testing as part of their betas – and expected people participating to do the same. Beta tests are wholly feeling like wasted time and energy for both participants and organizers.

The fun part of betas are the various personalities you meet. Some of them are very useful, for sure, and they seem more or less consistent in most of the betas I have done over the past three years (the personalities seemed a lot different 4+years ago).

1.) The Fanboy: Notice no ‘i’. I do not pick on these people. They found the game for them, and no matter what is missing it just fits their style. Their typical posts and response to posts usually cover how much they love the game, there really isn’t much to change, and if there are necessary changes they are sure the devs will get to it eventually. In the meantime they would pay for the product as is because no one understands how hard it is to make non buggy code anyway, and the devs made the game closest to what they need anyhow. Do not fight the Fanboy with reason or logic – they have their own. Niche gaming depends on them.

2.) The Doomist: The game is destined to fail without a complete code rewrite, mechanics change, graphics engine update, and firing of the entire development team. In fact – they won’t buy the game unless all of that happens. Them and their thousands of like minded friends. Devs need to listen to this guy, as they have participated in millions of beta tests (and “this” test always seems to be the worst one).

3.) The Undecided: Not on the game, no, but undecided on everything that is being discussed – yet they chime in. “That could be good, but so could this!”. “OOh, that is the best idea I have heard in a long time!”. “No wait, THAT idea is!”. They post a lot but still haven’t figured anything out, and have a hard time forming their own opinion. Easily swayed by shiny objects.

4.) The Quiet Observer: Watch out for this guy. He reads a ton of posts and plays hard. Doesn’t say much on beta boards, but knows is poop. When he does speak it usually carries the weight of a guy with 35 years of pent up anger ready to unleash on the world. Of course, he doesn’t kill anyone but typically makes a lot of sense. He is easily ignored by all due to the signal to noise ratio.

5.) The Cruncher: I love this girl the most. She breaks down every mechanic into something that is tangible – math. Of course, most devs cannot confirm nor deny those numbers, but they wade through combat logs and perform testing of such impact and use that if a dev company had a herd of these people their game would be perfectly balanced – for the number crunchers. They tend to ignore fun for facts, and when a person says that it “feels” like they swing their sword too slow, they promptly post [a*10 + b*1 ==> So (10a+b)**2 ==> 100a**2 + 2*10ab + b**2 ==> 100aa + 20ab + bb ] is the rate of swing for you race class combo and it is working fine. Crunchers tend to make everyone else feel stupid around them. Even if just by accident.

There are many more that make up the lovely beta test, but I have too much work to do to keep writing today.

If you are in your first Beta and it is in late stages  just remember this. The game you are testing is pretty much exactly how it is going to be with a little extra polish (hopefully) at launch. Not only can you not change the core concept, but the devs don’t want to do it anyway. Play the game as they want you to, submit your bugs, and decide if you are going to pass or play come release. If you know right off the bat you are not only not interested in buying the game, but don’t want to waste your time testing don’t feel bad about not participating either. You were chosen among a pool of hundreds of thousands, most likely by luck (or a specific spec your PC has) and the beta process isn’t what you really think/hope it is anyway. There will be a free preview weekend upcoming anyway where thousands more will get to trash the system. If you have better things to do than go do it.

That defeatist attitude my friends, is the result of beta fatigue.

17 Comments

  1. Tesh

    Or DIKU fatigue, as Scott over at Pumping Irony has noted. You really can’t keep doing the same thing over and over and convince yourself you’re having fun all the time. We like our routines, but we like variety, too.

    Reply
  2. Brian 'Psychochild' Green

    Betas have been about marketing for a long time now. By the time a large number of people are in the game, the course has already been set and it’s too late to make any changes beyond the merely superficial. The “real testing” took place with a much smaller group of people long before the unwashed masses were offered the ability to get in and see the shiny sights.

    The final stages of beta are calculated to get people interested enough in trying the game (hopefully), but not give them so much that they get tired of the game. You might be tired of all the different betas, but for a lot of people, this is their chance to try out the game before buying. After a bit of playing, their appetite should be whetted just enough to get them to buy the box. Showing them a half-finished game would be counter-productive.

    Reply
  3. Chris F (Post author)

    @Tesh: Yes, but not entirely. I pretty much know if I am stepping into the DIKU spotlight or not with the betas. It is definitely a part of it, but there is always hope something is interesting, fun, or innovative.

    @Psycho: I have been through over 20 betas at various stages (Alpha to Open)so I do understand that (depending on the phase). Some are great to work in (daily dev communication, focus tests, results oriented feedback) and others are just terrible. I think devs really need to make a statement of expectations when people are entering the beta – so the various personalities understand. I spend too much time reading too much garbage from “testers” on things that should just be stated on the outset.

    “we are in last stage of beta, this is how the game is going to be. Please comment on things that are still able to be modified at this point – abilities, graphics, etc.”

    in the past being a participant just felt more meaningful and useful for both the company and the tester.

    Reply
  4. Liam K

    I agree with Psycho, it is about marketing. It wasn’t always though. I’ve been a part of some beta tests where I felt the community had some serious effects on the games development. Anarchy online and the original UBO come to mind. Of course that was 8-10 years ago.

    Even the word beta has taken on new meaning. Hell WoW’s armory still brandishes the beta tag and it was released in 2007. Its a safety net, an excuse. And probably a tool to use on shareholders. “Look I know development is running long, but look at these ravenous fans crying out for a chance to play it!”

    I think betas are doing more harm than good to games now. For instance, without breaking any NDA’s, I will just say I will *not* be purchasing Champions Online.

    It seems to me, and probably to people who have been testing for over a decade, that alpha is the new beta.

    Reply
  5. Rog

    I agree, meaningful participation in a beta seems to have been lost.

    Games ship with the same bugs that persisted throughout beta, probably because they’ve recruited so many testers any bug reports get lost in the sea of “make me uber plz” requests.

    Without the marketing aspect, the betas would be smaller. Players accustomed to riding from free beta to free beta would balk, but I bet games would release with fewer bugs.

    What’s odd to me is the industry acts like they need these giant promotional pushes on their MMOs. What could be easier for word-of-mouth and viral advertising than an MMO? If you build a great game, they will come, absolutely.

    The giant betas are counter-productive IMHO.

    Reply
  6. Brian 'Psychochild' Green

    Rog wrote:
    What’s odd to me is the industry acts like they need these giant promotional pushes on their MMOs.

    The way things are currently structured, games do need this promotional push. It’s a great way to get free coverage for your game. Put out a Beta, get Fileplanet to carry your client, put out press releases, etc. People know what your game is without a lot of investment. Without the free beta, the large review sites are less eager to cover your game.

    Further, it’s the classic “call to action” marketing strategy: tell people there’s something free they can get, and they’ll go get it. This is much more effective than just having ads saying, “Our game is coming out soon!” If someone is willing to take a step to try something out, it makes them more likely to be willing to buy the product. “I might as well get the game, I already spent a few hours downloading the beta client that can be patched to play the full game….”

    If you build a great game, they will come, absolutely.

    This is demonstrably false. Any developer who believes this will fail, miserably. Simple logic: if you have a great game and nobody knows about it, nobody will play it. You need to do something to get attention, and a “beta” is a great way to get that attention.

    Marketing is vital because it works and developers ignore this fact at their own peril. From the figures I’ve heard, at least 30% of WoW’s initial development budget was for marketing alone. As much as people would like to believe WoW is wildly successful because it’s such a great game, the truth is not quite so simplistic.

    Reply
  7. Chris F (Post author)

    @Rog/Psycho: The issue with using Beta as a marketing tool is not “why” they are marketing, but for “what reason”.

    If your goal is to get 800K box sales, then lose 80% of your player base shortly thereafter then it isn’t a bad push. Heck, the initial money is grand, for sure. If you want to grow a long term, stable player base then it is a terrible way to do it. Any recent MMO release demonstrates this pretty well.

    EVE, I think, is the only game to come out of the gate slowly and perpetually build their long time subscriber base. I don’t even recall there being a beta for EVE – that is how under the radar it was. Yet there they are, with their “A” design team in tact, building away slowly. It has proven to be a good business model for them.

    Reply
  8. Rog

    I’d like to point out that both The Sims and the original Half-Life were sleeper hits with absolutely zero promotion prior to release. The Sims was given a shared display at E3 (hell I saw it with no lineup and nobody waiting behind me for a demo either) and Valve was relegated to an off-site booth.

    Note also that Sony / 989 spend most of their time promoting sports games, Everquest wasn’t even shown publicly, besides a poster.

    Then there’s the huge success of CounterStrike and Defense of the Ancients as mods building interest around communities. Or how about Doom or Warcraft, sold as shareware. Portal anyone?

    The biggest selling games, especially on the PC, have had almost no pre-release hype. Not counting sequels.

    I’m not saying marketing after release isn’t necessary, it’s just gotten overblown where it’s just useless spinning of wheels prior to release. Arguably, it does damage as it burns out the playerbase before the ticketsales. Imagine movie trailers where you get to see the whole movie in production.

    Open Betas are take-a-general-piss-in-the-direction-of-the-wall marketing.

    This happens because either they have no clue about marketing, or they’ve got too many boys in suits trying to prove they’re necessary by over-marketing. Convince everyone pre-release hype is necessary and they’ve cemented their jobs as magical moneymaking gurus.

    It’s bullshit. They should be spending this time, energy and money on their game.

    @Chris F: Yes there was a beta for EVE but it was more of an actual Beta. The game was promoted heavily in an online campaign while they were still with SSI, but I think most would agree that EVE’s playerbase has been built slowly on reputation.

    Even WoW, had a limited, non-open beta. Mind you, they effectively sniped the raiding guild crowd from Everquest and the expectations from the existing Blizzard fanbase didn’t hurt either. Blizzard is an example of much tighter, restrained and targeted marketing strategies, at least in getting where they are now. They can afford to throw it around sloppy at this point, but anyone copying their current strategies would be making a mistake because first they depend on getting where Blizzard is now.

    Reply
  9. Rog

    Marketing is vital because it works and developers ignore this fact at their own peril.

    I’m not saying that marketing isn’t necessary. I’m saying it’s overblown and that the concept of building hype prior to release has particularly gone off the deep end. They’re blowing their wad too soon.

    From the figures I’ve heard, at least 30% of WoW’s initial development budget was for marketing alone. As much as people would like to believe WoW is wildly successful because it’s such a great game, the truth is not quite so simplistic.

    You’ve got that one slightly wrong, that’s first year of expenditure. Prior to release, WoW had no ad open campaign and barely even a “coming soon” box / signage in stores. Blizzard did have their grass-roots sniping going on, but that’s exactly what I’d call appropriate pre-release marketing, without the hype.

    I stand by what I’ve said, because there’s so much interest in the MMORPG genre right now and the playerbase is very savvy:

    A great game would shine through and it could barely be mentioned and yet still masses would come on word of mouth.

    At the same time, a not-so-great game may get an opening week splash through heavy pre-release hype, but within the year its playerbase will turn sour and the CEO will depart while they try to rescue the game. Need I bother giving a couple of examples?

    Far, far, far more depends upon the game than the hype.

    Reply
  10. Brian 'Psychochild' Green

    Chris F wrote:
    EVE, I think, is the only game to come out of the gate slowly and perpetually build their long time subscriber base.

    The thing people forget when looking back through the lens of time is that EVE didn’t just “come out of the gate slowly”, it failed. The original publisher bailed on the game; under these circumstances most games would have simply failed and become a footnote in history. As far as I understand it, CCP got some assistance from the Icelandic government and was able to keep the game running. Sure, they’ve grown and even thrived since then, but it’s not a path other companies can easily take.

    Rog wrote:
    I’d like to point out that both The Sims and the original Half-Life were sleeper hits with absolutely zero promotion prior to release. The Sims was given a shared display at E3[…]

    Perhaps we have different definitions of “promotion”, but getting coverage at E3 qualifies under my definition. These games may not have had full-page glossy ads in major magazines, but they certainly got promotion.

    The Sims is also a funny case because EA management didn’t think it would succeed. I talked to Will Wright in person and he backed up that story. The only reason the game got released was because Will had a secret team work on the project without EA’s knowledge. He presented a complete game to them, they published it expecting it to wither and die, and were surprised when it did otherwise. I also remember that it did get some promotion, because everyone was talking about the “revolutionary A.I.” the game displayed. This is how they got the hard-core to buy it and get it into the hands of what ended up being the real audience: their better halves.

    And this goes right to the heart of my argument. A beta is cheap promotion. It’s not a sure-fire way to get people to buy your game, but it’s “testing” game companies should be doing anyway and it’s a way to get your name out. Even if WoW didn’t spend a dime before launch, they had a Beta that got people whipped up into a frenzy. If WoW was such a wonderful game and great games do well automatically, why did Blizzard feel the need to spend millions upon millions of dollars on marketing, even if it was after launch?

    A great game would shine through and it could barely be mentioned and yet still masses would come on word of mouth.

    Really? Then answer me this: where’s my Beyond Good & Evil sequel?

    Reply
  11. Chris F (Post author)

    @Brian: I didn’t realize EVE had failed at one point. Thanks for the history lesson! As far as I knew it just started off slow before gaining popularity.

    In This old post I wonder along the lines of what Rog is getting at. Take a game, focus on building that game, and when it is feature complete and actually playable, THEN market the game. All the money and time spent in pre-marketing a product that you don’t know the core of (what classes? How many capitol cities? PVP? What kind?) can be avoided. The mass hype machine – especially in MMO’s – seems to be doing more harm than good. Plus they blow a lot of cash on it, and then realize they don’t have enough to finish the game.

    Take SWTOR for example – there is HUGE hype about the game right now, probably 2 years out. Imagine if they actually finished the game before mentioning it? Devs would know what the game had (instead of what they are hoping to be able to put into it before launch) and the hype machine would hit full stride in no time at all – instead of having to create its own hype.

    Because of the long development cycles it seems games are the only product that focus more on the prehype of the game than any other industry. You don’t hear about what kind of flavours of soft drink *might* be coming out next year, you hear about it when it’s launched. Movies pre-hype – but the movies are already shot and edited by the time the commercials come out. Music doesn’t do it until there are clips of songs to release. Even then, you know what you are getting.

    Reply
  12. Rog

    Brian ‘Psychochild’ Green wrote: Perhaps we have different definitions of “promotion”, but getting coverage at E3 qualifies under my definition. These games may not have had full-page glossy ads in major magazines, but they certainly got promotion.

    You’re drifting though, there’s promotion and there’s pre-release marketing blitzes. Will Wright having to apologize while they wait for a shared machine to become free just to show the game barely even counts as promotion. There was no marketing budget involved in that.

    It’s not a question of perspective, it’s that you’re responding defensively in favour of marketing (business school graduate?) without regard for context. I’m not saying promotion isn’t involved, I’m saying some promotions get in the way of development or waste money when development isn’t even completed yet (and likely could use the wasted funds).

    And this goes right to the heart of my argument. A beta is cheap promotion.

    Cheap from a marketing perspective maybe, but expensive when it’s getting in the way of the final months of development and bug fixing.

    Maybe if they did these huge betas after the game goes gold, it’d make sense on a cost / reward perspective, but they’re wasting a lot of time, energy and yes, plenty of dollars rolling out server structure and support before taking in a single buck. If it was ~actual~ testing, it would make sense, but that’s the whole subject isn’t it? It’s not really testing.

    Could you stop and step out of the marketing shoes for a minute and see it from a development perspective?

    The problem is you need to hatch the egg first before you sell the chicken. Game development comes first.

    Really? Then answer me this: where’s my Beyond Good & Evil sequel?

    Um, currently under development. Regardless, that’s not exactly an MMO with masses waiting behind it. You’ve quoted me completely out of context… again.

    Reply
  13. Rog

    @Chris F: That’s pretty much what I’m saying. Right now in the games industry, I’d say the priorities are a bit screwed up.

    The business is making games, then selling them. Not necessarily in a strict timeline, but priority-wise that’s often the same.

    Reply
  14. Rog

    And I don’t mean to be so hard on Brian, I know he’s been in and around development enough to know what I’m talking about.

    I think, unfortunately, it’s the nature of marketing to justify itself, to promote more marketing. It’s not an eternal tap though. There’s a threshold of too much and that’s just my opinion here, that it’s gone over the top.

    Reply
  15. Tesh

    Chris, I can’t help but think that early promotion of SWTOR is partially a way to drive advertising revenue to bolster development budgets, as well as get a bead on what may or may not fly with potential customers, all the way from content to the eventual consumer business model. I’ve certainly opined about what I’d like to see from the game, and it may still be early enough to make a difference in that game. It gets the fanboys in a lather, but it might be a savvy shot in a smart project manager’s quiver.

    As for a “beta proper”, yes, I see them as late-project marketing buzz with a twist of unpaid QA stress testing. I don’t expect to make a difference on the game as a whole, but I can maybe suggest a few bug fixes and assess whether or not I’m going to be excited about it and promote it to friends. *shrug*

    Reply
  16. Chris F (Post author)

    @Tesh:

    Interesting.. isn’t marketing a straight line expense? Revenues to follow. They don’t use ads on their webpage unlike other MMO god-types (jab at our good friend WoW). Hype comes at an expense that doesn’t turn into profit until a sale is made. I would suspect they already have a giant budget, and considering the IP and scope of work wouldn’t need page hits to justify what budget is needed.

    I would also argue that the design for the game is pretty much done. How it will play, options, etc are all crystalized at this point. For example, if it became overwhelmingly obvious that MMO players DIDN’T want a story-driven MMO they aren’t going to change that. Sure there is room on some fluff for change, but the core is already where it will be upon launch. I just can’t see any meaningful change gleaned from forum posters and commentators considering no one know what – or how – the game will play like to begin with (besides those in the inner circle with their own vision).

    That being said it seems like a typical major hype effort.

    Reply
  17. ixobelle

    wtf is up with Aeria games? I get like 17 beta invites a day with those guys. It makes seeing actual beta invites in my inbox reach for the spam filter, and then be like OH! HEY! THAT’S A REAL ONE!

    Reply

Leave a Reply

%d bloggers like this: