Jump to content

What is the true purpose of the PTS ?


supertimtaf

Recommended Posts

Have you ever heard the tragedy of the 'Great Bear Nerf'?

 

A long time ago, in a galaxy far, far away, a game called 'World of Warcraft' released its first expansion, and there was much rejoicing as it finally offered support for end-game roles other than the one you basically had to play in the initial release. For Druids, this meant they could finally effectively tank (most*) raid bosses in 'Bear Form' instead of being required to heal. And things were good for a little while...

 

Then came the (not incorrect**) chorus from the forums 'Bear Druids do too much damage and take none in return' and the developers deployed a 'damage nerf'' on the test servers. Players flocked to the PTS and were basically unanimous in their evaluation: 'Nerfed damage Bears can not maintain aggro, making them horrible tanks.' After a couple of weeks of testing, the 'nerf' patch' was deployed on the live servers and people discovered that, oddly enough, Bears couldn't tank effectively anymore.

 

There was a happy (and somewhat embarrassing) conclusion to this tale: Literally within 24 hours of the nerf being rolled out, Blizzard deployed a 'hot fix' to give bear form a 'threat modifier' like other tanks had, allowing them to effectively hold aggro once again.

 

The speed with which this hotfix was deployed indicates to me some combination of the following:

  1. The 'fix' was really that easy to apply but the developers didn't believe it was actually necessary on the PTS, and/or
  2. The developers knew the issue was real but weren't allowed to 'fix' it in the patch for some reason***, so they wrote the 'hotfix' code ahead of time so it could be deployed quickly once its necessity was blatantly obvious.

 

So, developers not acting on PTS feedback is by no means exclusive to Bioware: I am generally of the opinion that the PTS is intended more as a 'preview / advertisement' than as an actual 'does this actually work' environment.

 

 

*There were at least a couple of 'bears need not apply' fights based around mechanics or gear Druids didn't have access to: I think it took another expansion before bears were really on an equal footing with the other tanking specs (Full Disclosure: My 'main' in WoW was a bear tank and I seriously considered quitting the game when it was plain the nerf was going to be deployed, but with the hotfix I kept on tanking though the eventual cataclysm).

**Bear Druids definitely needed tweaking at the very least, but I think the major issue was our dominance of warriors in PvP due to having both good damage output and huge physical armor ratings that made incoming physical attacks fairly ineffective.

***Whenever a class was nerfed, there was always a "well, I guess we know which class the 'guys in charge' couldn't beat in the battlegrounds last week" forum comment. I don't know how accurate these comments were, but there was always a feeling that certain classes were heavily favored by whoever made the final decisions on buffs/nerfs.

Link to comment
Share on other sites

*snip*

 

An interesting tale that I didn't knew about tbh. One different thing however was that in both ways, the hotfix arrived soon enough. Here, even bugs tracked on the PTS aren't fixed on time for the release, and some of them won't be fixed until a month or so, no matter how gamebreaking they are (ahah, passives increasing stacks ? Nah, won't be fixed). Heck, even the WIP icon on amplifier window was ported straight to live.

 

To be honest I'm quite envious of Bioware's dev team. It must be pretty wonderfull to know that no matter how much you mess up a live release, you will never risk your job, no matter how many bugs you introduce and features you don't implement correctly. Even while working on the same line of work as them, I would lose my job after two terrible patch releases like these ones if I don't fix them in a week.

Link to comment
Share on other sites

i think that the PTS needs to have a new function then only finding bugs and problems in the new patch's that there need to fix.

 

i think really that the communety most look if the new things or chance's there going to make is something good what the communety can accept.

that if something is wrong like we see with the New Amplifier Pane that if its looking bad in the game like its taking to much space on the screen and has no collapse option that something like that needs to be fix in the PTS since if its so looking back and there give notting about the feedback it has what is only bad feedback then you know its only become worse when you release it on the live servers that the feedback is only getting worse.

 

i know some game's that have also a PTS and that players can test on it the new things before it go's on the live servers.

and there its not only about looking for bugs and problems but also if the new things are good and need maybe a small chance here and there before it go's live and the developers are listing to then and chance it if a lot off people hate it.

 

for one side its no suprice that bioware not has listing to the bad feedback about the new amplifier pane since bioware is doing that for a long long time all more not listing to the communety.

if bioware was listing to us then a lot off bugs and problems have been fix all.

but sadly there are not doing it since we all know there give no crap about us what we think and then you get a 12 page long bad feedback only about 1 thing that shows the communety hate's it.

 

and you get then also that communety members going to hate then for not listing to the communety and there problems.

Edited by Spikanor
Link to comment
Share on other sites

...

Heck, even the WIP icon on amplifier window was ported straight to live.

 

To be honest I'm quite envious of Bioware's dev team. It must be pretty wonderfull to know that no matter how much you mess up a live release, you will never risk your job, no matter how many bugs you introduce and features you don't implement correctly. Even while working on the same line of work as them, I would lose my job after two terrible patch releases like these ones if I don't fix them in a week.

 

It is quite baffling. Just for context here, rather than any kind of silly boast: I work as a lead software engineer at one of the biggest companies on the entire planet, and as-such, I really can't understand what's going on here at all. The overwhelming majority of developers / QAs that I've worked with have been really passionate about putting out high-quality work - it's a cultural thing - sure, sometimes business pressures get in the way and things aren't quite perfect, but on the whole people do strive for quality.

 

I obviously have no insight to what's going on at BW these days, and whether it's lack of resources or apathy / incompetence I just don't know, but how things like the 'WIP' icon and the many other howlers / own-goals get through to live on a regular basis is, as I've said above, baffling. I know I or my team would never survive such a history of poor releases, not that we would ever get to that stage.

 

I've been playing for a while now and I've put in more hours that I'd comfortably admit to, and I've always enjoyed the game - fundamentally it is a good game. However, as the months go by though I find that I get more and more irritated at myself for continuing to pay for a product which is deteriorating so rapidly through an apparent lack of care or interest.

Link to comment
Share on other sites

The overwhelming majority of developers / QAs that I've worked with have been really passionate about putting out high-quality work - it's a cultural thing - sure, sometimes business pressures get in the way and things aren't quite perfect, but on the whole people do strive for quality.

 

As an interesting story, we had a patch one day that got released... Well, not for everybody. Some peoples had the patch to download, and others did not. Players who didn't had to download anything were playing fine and could log back in with no issues. I and many others however, who had to download said patch, would find our game just... Well just broken. Bugs, glitches, visual issues... Nothing was working properly. You couldn't do anything.

 

After this day, after a little investigation from Bioware, we received a message from Keith Kanneg, producer of Swtor at the time (I don't know if he's still there, we may have lost contact since a few years. :p ) to apologize for the weird issue, and telling us that from that point on, there would be a QA team assigned to patch releases, to make sure it worked.

 

So there are two issues now with said statement. First one is that well... Swtor didn't had any QA check before releases, meaning that most bugs weren't checked or even known from the devs when a patch was pushed live. Same as every issues around.

 

Second issue is that now we're supposed to have a QA team dedicated to... Well. Avoid this kind of scenarios. I'm not saying that a QA team will identify everything, sure. But I still expect that major bugs (passives not applying) or even the small WIP icon on amplifiers window would have been found out by said QA check, as those were immediatly spotted on by players. So what happened ? Do they just launch the patch on their test server, log in to see if the game doesn't crash and call it a day ? I hope that's not the case.

 

But at the same time I do hope so, as this seems like some good job opportunity for many peoples who lost jobs due to covid. Doesn't require much qualifications. :D

Edited by supertimtaf
Link to comment
Share on other sites

But at the same time I do hope so, as this seems like some good job opportunity for many peoples who lost jobs due to covid. Doesn't require much qualifications.

 

If Bioware actually fixed the bugs reported on the pts before going live, more people would still be testing. It would also help if you could download the patches faster and while you play the live game.

 

They wouldn’t even need to pay new people because we were doing it for free. Sadly, when you report bugs in detail and they still push them live, there is no incentive to help them find said bugs on the pts. That’s what I mean by Bioware wasting a valuable resource.

 

The other thing to consider is they might be paying a team to QA, but the devs don’t have time to fix the bugs before the schedule says they have to push the patch out. Or the QA guys report them and it doesn’t make it to the dev team (sort of like our pts reports don’t seem to make it either :rolleyes:)

 

There is one other thing to consider, Covid. How many Bioware staff are affected by it? They are also mostly working from home as far as I know. So things could be getting lost in communication as well. It’s hard to manage a team remotely and it’s harder if you’re understaffed or not feeling great too. (I’m not making excuses for them, just some points to consider).

Link to comment
Share on other sites

If Bioware actually fixed the bugs reported on the pts before going live, more people would still be testing.

 

They do indeed fix the bugs , but the overriding perception (and let's honest, 'perception' is all that matters nowadays, sadly) is that they somehow intentionally "ignore" flaws-in-their-code on purpose.

 

As such, i totally get what you're saying and i agree with your point. Furthermore, imho it's BioWare to blame for said perception, since it SEEMS as if they either don't read or worse don't care to acknowledge certain "obvious" bugs that make it to live.

 

The solution to this type of perception spiral , imo, is more of that word most corporations hate: Transparency.

 

More transparent type posts like THESE: https://www.swtor.com/community/showthread.php?p=9889956#edit9889956 ...for example.

 

But, for whatever reason, we don't get that very often around these umm official forums. Instead, not always of course but for the most part, we get *static* . Probably, cuz the Devs are too busy coding to have time posting. Which is totally understandable , but nevertheless disappointing.

---

The other thing to consider is they might be paying a team to QA, but the devs don’t have time to fix the bugs before the schedule says they have to push the patch out.

 

Yeah that ^ is mostly likely the situation, by & large.

 

I worked QA for 5 years on another 'STAR WARS' game and not to stereotype too much here but: In my experience, Devs (coders) truly want to fix all the bugs all the time....but there just simply isn't enough time.

 

So, certain things get prioritized or "shelved". Or, unfortunately, forgotten about. Plus, very often some bugs---strangely---don't even show up until they hit the LIVE servers. (*Whenever that happened, we at QA always felt soooo awful. And of course we were the ones who always got blamed too, even though technically it wasn't anything we could've foreseen on the Test Server)

 

----

Or the QA guys report them and it doesn’t make it to the dev team (sort of like our pts reports don’t seem to make it either )

 

This ^ however is highly unlikely. BioWare definitely sees & reads ALL reported tickets (eventually) .

 

And they certainly monitor the PTS forum threads.

 

But yet, still the aforementioned perception exists that they don't.

 

Perhaps someone who seems to think having official BioWare Dev interacting constantly with the "community" over on some 3rd-party platform instead of here on the actual game's website is somehow better/easier could go *ping* ChrisSchmidt directly and ask him about the reality of SWTOR's QA Team? And then report back to us here at the forums? :cool:

Edited by Nee-Elder
Link to comment
Share on other sites

 

...

 

There is one other thing to consider, Covid. How many Bioware staff are affected by it? They are also mostly working from home as far as I know. So things could be getting lost in communication as well. It’s hard to manage a team remotely and it’s harder if you’re understaffed or not feeling great too. (I’m not making excuses for them, just some points to consider).

 

Well, yes, that's certainly possible. However remote development was a thing way before Covid - it's not at all unusual for devs to WFH (*) - I've spent a fair bit of my worktime doing just that, particularly in recent years as this way of working gets ever-more popular. And, yes, I realise my team isn't BW's team, but I manage a team of entirely remote developers - some not even in the same continent - and we manage perfectly well producing quality software that is use by a hell of a lot of people. Covid makes for extra challenges, sure, but personally I believe they are manageable ones

 

* Sometimes working from home is the only way you can actually get stuff done without distractions :D

Link to comment
Share on other sites

But at the same time I do hope so, as this seems like some good job opportunity for many peoples who lost jobs due to covid. Doesn't require much qualifications. :D

 

Neither does being snarky in the forums funnily about other people's work, its quite easy and requires nothing more than a keyboard:rak_04: And you people wonder why the devs don't spend more time here.

Edited by FlameYOL
Link to comment
Share on other sites

The other thing to consider is they might be paying a team to QA, but the devs don’t have time to fix the bugs before the schedule says they have to push the patch out.

 

This is most likely it, though of course I don't work on game development so I'm just guessing becase on the perception as a player. We know they don't have a huge development team, so its likely that ends up leaving more bugs than its possible to fix before they have to release the patch. Though the better question for me would personally be if they have a deadline from EA or if its more of an internal decision from Bioware Austin, though its unlikely we'd ever have an official answer in these forums.

 

I do think this patch was likely rushed out the door due to already being a few weeks late due to the weather storms that happened in Texas, meaning they'd rather release it now and fix it later than going even longer without the patch... if that was the case, I wouldn't have minded waiting. Though I understand that is not a position everyone took.

Link to comment
Share on other sites

This is most likely it, though of course I don't work on game development so I'm just guessing becase on the perception as a player. We know they don't have a huge development team, so its likely that ends up leaving more bugs than its possible to fix before they have to release the patch. Though the better question for me would personally be if they have a deadline from EA or if its more of an internal decision from Bioware Austin, though its unlikely we'd ever have an official answer in these forums.

 

I do think this patch was likely rushed out the door due to already being a few weeks late due to the weather storms that happened in Texas, meaning they'd rather release it now and fix it later than going even longer without the patch... if that was the case, I wouldn't have minded waiting. Though I understand that is not a position everyone took.

 

I think you're on to part of what is going on .. but IMO this is not it entirely. Some of what we see was clearly PRE-Covid. Some of what we see is unquestionably "perception" rather than fact. Some of what we see is fact

(not fiction).

 

The real unfortunate part is that a wall has been built between the community and the team. And that's ashamed really.

 

As players we need to try to see through their eyes when a company is dealing with multiple disasters (covid and the winter storm situation).

 

As well the team would do well to listen carefully to those who are trying to help in PTS.

 

NO... a representative can NOT answer to every single thread that is being posted. That's just silly to even begin to imagine such non sense. On the other hand there is something to be said about "taking appropriate action". The new UI is such an example.

 

** if the team had proceeded with the new UI scaled differently with the same statistical information (what use to be on two pages) AND a collapsible table with all of the amplifier listings this would not have erupted on nearly the scale as it has.

 

** some will complain regardless. [/shrugs] what can I say. It's unfortunate.

 

** Communication: I'm not ready to "burn down" the entire forum board just yet. Though my involvement might diminish from time to time. Just because this place can do just as well (if not better) without me doesn't mean it's THAT bad !! (but communication issues are meant for another thread ... at another time. Not here !

 

I'd rather tear down the wall and build a bridge !!

Edited by OlBuzzard
Link to comment
Share on other sites

The other thing to consider is they might be paying a team to QA, but the devs don’t have time to fix the bugs before the schedule says they have to push the patch out. Or the QA guys report them and it doesn’t make it to the dev team (sort of like our pts reports don’t seem to make it either :rolleyes:)

 

What sadly doesn't help me believing that the QA team does much is the fact that we have a "Known issue - Patch 6.XX" at every release. This is normal, and is prepared in advance usually to list the numbers of bugs identified by QA just before release. Every MMO or live service does this tbh.

 

What's bothering me however is that while said thread exists, BW's post gets filled at more than 80% by bugs notified by players. So most of what BW places in this thread is mostly discovered by players. Too much in my opinion to justify that the QA team is working properly. Even bugs notified on the PTS and not fixed for the release are not notified here until somebody complains about it too much on the live release.

 

It may very well be stuff that has gone forgotten. That I know can happen. But you usually forget once or twice. Not systematically.

Edited by supertimtaf
Link to comment
Share on other sites

a very small portion of the player base uses the pts, an even smaller one uses the pts forum to post feedback and only a handful of people write feedback that's better than "its bad but I don't know why". the new interface is fine, people generally dislike change and it will be the echo chamber opinion for a while until people get used to it, most likely an option to collapse the amplifier window is in the works so I wouldn't worry too much for now

 

most likely the true purpose of the pts is just like the forums in general but for testing, a place where ppl can just post their opinions, suggestions etc. in their pursuit of approval :D

Link to comment
Share on other sites

a very small portion of the player base uses the pts, an even smaller one uses the pts forum to post feedback and only a handful of people write feedback that's better than "its bad but I don't know why". the new interface is fine, people generally dislike change and it will be the echo chamber opinion for a while until people get used to it, most likely an option to collapse the amplifier window is in the works so I wouldn't worry too much for now

 

most likely the true purpose of the pts is just like the forums in general but for testing, a place where ppl can just post their opinions, suggestions etc. in their pursuit of approval :D

 

Incorrect.

 

If you have followed the PTS forums at any length of time at all you would know that several people are very good at what they post. Some provide detailed information. Granted there are not as many who provide spread sheet information as some did (pre 6.0) when crafting took a serious turn for the worse. (Yes ... some have provided detailed spread sheet data/analysis).

 

In any event ... the simple fact is that those who have actually participated in the PTS have became increasingly disillusioned with the "stated purpose" of the PTS since the vast majority of factual information and REQUESTED...feed back has been promptly ignored.

 

IMO ... this sort of thing builds walls ... not bridges !!

Edited by OlBuzzard
Link to comment
Share on other sites

I think you're on to part of what is going on .. but IMO this is not it entirely. Some of what we see was clearly PRE-Covid. Some of what we see is unquestionably "perception" rather than fact. Some of what we see is fact

(not fiction).

 

The real unfortunate part is that a wall has been built between the community and the team. And that's ashamed really.

 

As players we need to try to see through their eyes when a company is dealing with multiple disasters (covid and the winter storm situation).

 

As well the team would do well to listen carefully to those who are trying to help in PTS.

 

NO... a representative can NOT answer to every single thread that is being posted. That's just silly to even begin to imagine such non sense. On the other hand there is something to be said about "taking appropriate action". The new UI is such an example.

 

** if the team had proceeded with the new UI scaled differently with the same statistical information (what use to be on two pages) AND a collapsible table with all of the amplifier listings this would not have erupted on nearly the scale as it has.

 

** some will complain regardless. [/shrugs] what can I say. It's unfortunate.

 

** Communication: I'm not ready to "burn down" the entire forum board just yet. Though my involvement might diminish from time to time. Just because this place can do just as well (if not better) without me doesn't mean it's THAT bad !! (but communication issues are meant for another thread ... at another time. Not here !

 

I'd rather tear down the wall and build a bridge !!

 

I don't entirely disagree with the sentiment of building bridges instead of building walls, sadly as long as I have played Bioware has had a communication problem. They don't tell us what's coming too ahead of time, at best we know what's coming a few months instead of the entire year, though sometimes they can be better at it like when they explain the thought process behind their decisions, such as one thread explaining how fixing bugs wasn't easy, I really liked the new uprising interview with one of the developers too ... but at the same time, I don't blame them for not choosing to engage with these forums. But I'll end the thread here, I was about to post a huge textwall but I'll stop here, hopefully communication and the forums atmosphere will be better in the future. Though I doubt it'll happen, but hey I'd be glad to be proven wrong.

Link to comment
Share on other sites

I don't entirely disagree with the sentiment of building bridges instead of building walls, sadly as long as I have played Bioware has had a communication problem. They don't tell us what's coming too ahead of time, at best we know what's coming a few months instead of the entire year, though sometimes they can be better at it like when they explain the thought process behind their decisions, such as one thread explaining how fixing bugs wasn't easy, I really liked the new uprising interview with one of the developers too ... but at the same time, I don't blame them for not choosing to engage with these forums. But I'll end the thread here, I was about to post a huge textwall but I'll stop here, hopefully communication and the forums atmosphere will be better in the future. Though I doubt it'll happen, but hey I'd be glad to be proven wrong.

 

I agree !!

 

It is unrealistic to expect a constant explanation or reply to "all things said" on the forums !! That just does not make any sense. Yet there is something to be said about responding "in kind" when dealing with the PTS (at least on occasion). If there is a sharp disagreement on the overall objective then perhaps some sort of "additional input" (????) is needed to help the situation before it goes thermal !

 

A short summary: it is without question an "unrealistic expectation" for the development team to spend valuable time "responding" to everything that is going on !! IT IS a VERY good thing to build a better understanding especially to those who are testing and giving back requested input.

 

If said "response" is to simply "go with what we have regardless" ... that is communicating ! It's not a good way to communicate ... but it is communicating none the less. This IMO just adds another brick in the wall !

Link to comment
Share on other sites

×
×
  • Create New...