except that 9/10 times game metrics are not representative of what is actually going on.

Not having a go, but it did make me laugh to use imaginary statistics to back up an argument that statistics are useless.

Personally, I find it reassuring that devs are trying to find and use hard data to make decisions based on reality rather than instinct and rumour (also known as ignorance and prejudice). If they went by what is being shouted about on the forums it would be like changing government policy day to day depending on whatever today's tabloid fad was.

The trick is in the interpretation. They have to decide which figures tell them something useful, and what the figures are actually telling them. It's easy to get the wrong end of the stick. The hypothesis given above that (maybe, don't have actual proof I'd point out) dps mercs/commandos could look like they're performing better because only skilled holdouts are still playing them. Maybe you could look at the valour (for pvp time played on a char) to see if the high-scoring mercs are using experience to match noob sents.

Point: attempting to base decisions on reality rather than prejudice = good.

Not having a go, but it did make me laugh to use imaginary statistics to back up an argument that statistics are useless.

 

Personally, I find it reassuring that devs are trying to find and use hard data to make decisions based on reality rather than instinct and rumour (also known as ignorance and prejudice). If they went by what is being shouted about on the forums it would be like changing government policy day to day depending on whatever today's tabloid fad was.

 

The trick is in the interpretation. They have to decide which figures tell them something useful, and what the figures are actually telling them. It's easy to get the wrong end of the stick. The hypothesis given above that (maybe, don't have actual proof I'd point out) dps mercs/commandos could look like they're performing better because only skilled holdouts are still playing them. Maybe you could look at the valour (for pvp time played on a char) to see if the high-scoring mercs are using experience to match noob sents.

 

Point: attempting to base decisions on reality rather than prejudice = good.

that was kind of my point, and hence why i posted that Mark Twain quote. statistics are the devil.

you are right tho. one single metric does not provide any kind of a clear picture. you have to look at all metrics, develop some good crosstabs, and then try to understand what is really going on based on how the metrics are related/unrelated.

I am not surprised at the use of statistics. They've been doing this for some time.

What I AM surprised at is how readily Jonathan Crow has admitted they are STILL using those metrics for PvP balance. Now. In an MMO with some of the worst class balancing I've seen in a LONG time. Balancing which seems to be getting worse patch by patch.

It's like sitting on the prow of the Titanic talking about how she's practically unsinkable as she slips beneath the waves.

Full speed ahead Mr Boatswain, full speed ahead!

This is me thinking here- if I had a game that was honestly balanced, and I honestly felt like my metrics supported my balance- I'd be happy to show it because it would quell a large chunk of the unrest. It's hard to support crying nerf or buff for classes when you have available stats that disprove the people asking for those things. However- when there's no stats you're stuck with everyone spouting opinions which are supported at best by skilled play, or at worst by 'last WZ I got pwnd by this class NERF!!!'.

This assumes the unrest is caused by people capable of recognizing that statistics disprove their point. IMO you're overestimating the average IQ level of the unrest-causers.

This assumes the unrest is caused by people capable of recognizing that statistics disprove their point. IMO you're overestimating the average IQ level of the unrest-causers.

Except statistics will at least reduce that to the low IQ folks- those who know the game will be able to have more meaningful conversations. Right now- everyone is at each other's throats and there's never any ground gained because it always comes down to opinions, often biased ones at that.

It doesn't help that BW is straight out lying about what their metrics are showing- nobody believes them when they say the KDR is relatively stable across the board, how stupid do they think we are? Or mercs- I think nobody is going to deny they have the lowest rep in RWZ... and it's nothing new, they've been in this position well over half a year. Where is BW on these things?

Some players are never going to be happy, respectful, considerate or patient. They blow things out of proportion, deal only with absolutes and will never say a positive thing about anything or anyone unless it agrees completely with their own position. These players, many of the posters above, are good at venting and that is all.

I'll take BW's analytics over most of the posters on this board any day of the week. But I'll take numbers I can gather/monitor/verify and analyze myself over anyone's at BW. The main problem is that I/we have access to very few of those numbers. And another big problem is that many people have opinions they will defend to the death based on no verifiable facts (opinions like: "the world is flat" and "ur is the best defensive ability in the game" and "sorcs suck at team play" and "maras are overrunning the game" - even though there are far more sorcs than anything else in the 21-50 range and @50 on both POT5 and Jung Ma last time I checked).

Some players are never going to be happy, respectful, considerate or patient. They blow things out of proportion, deal only with absolutes and will never say a positive thing about anything or anyone unless it agrees completely with their own position. These players, many of the posters above, are good at venting and that is all.

 

I'll take BW's analytics over most of the posters on this board any day of the week. But I'll take numbers I can gather/monitor/verify and analyze myself over anyone's at BW. The main problem is that I/we have access to very few of those numbers. And another big problem is that many people have opinions they will defend to the death based on no verifiable facts (opinions like: "the world is flat" and "ur is the best defensive ability in the game" and "sorcs suck at team play" and "maras are overrunning the game" - even though there are far more sorcs than anything else in the 21-50 range and @50 on both POT5 and Jung Ma last time I checked).

So...i will start with a link to the QA in question- http://www.darthhater.com/articles/swtor-news/22564-jonathan-crow-answers-questions-regarding

 

 

In particular in want to highlight the following; "We pay very close attention to both engagement and balance in PvP. Some things we examine are player kill to death ratios and average rewards gained for every class to make sure that balance is maintained across the board.

 

And ; Balance-wise, kill/death ratios are well-balanced, with all Advanced Classes having kill/death ratios very close to each other.

 

I guess that means that dps sorc/sage and dps merc/trooper is perfectly fine? What about Shieldtech PTs? According to these metrics, "kill/death ratios are well-balanced" across all AC...

 

For real?! I simply cannot comprehend how this could possibly be true when you have some classes who clearly have great difficulties to perform on a adequate level. Hell, even i am getting interested in rolling something like a shadow these days...

Suddenly it all makes sense.

Now, I mean, in a shooter or something, it would make some semblance of sense to do it that way. Wouldn't be a perfect way to do it. But it might be better than throwing darts on the wall.

But a "kill" in a shooter isn't awarded the same way as it is in SWTOR. I'm not a "gee looky at my kills and medals at the end of a warzone" guy, so I never learned exactly what constitutes a kill in SWTOR. But it sure as Hell isn't a deathblow, which is how shooters do it. Couting up the amount of own-team kills vs the other team's deaths frequently yields something like a 4-1 ratio.

From what I can tell (and again, I'm someone who's preoccupied with winning and couldn't care less about stats or medals, so I might very well be wrong), a kill in SWTOR is the equivalent of an assist OR a kill in a shooter. Maybe more stuff. Seem to get quite a few kills on my healer, and its not like I go around dotting everything before I start tossing probes out or use grenade on cooldown (out of fear of accidentally breaking someone's cc when my attention's divided between the warzone and my op frame). So is healing a guy who kills a bunch of people giving me kills? If I mez someone, then someone else kills him a minute later when I'm nowhere near the guy, do I get a kill despite not damaging him?

Here's where the problem lies. Let's take Halo, because that's what I've been playing lately. Suppose there was a map with a ton of choke points. Suppose there was a weapon with a huge splash area that did very little damage. And suppose shields were 3 times as strong and regenerated ~ every 30 seconds. And suppose assists were counted as kills. Kill/Death ratio would have the devs believe that weapon was way better than, say, a DMR or battle rifle. But they'd be horribly, horribly wrong.

That's not unlike what the data in swtor would lead them to think if they use kill/death when a kill isn't actually a kill. Honestly, it would be better to use solo-kill/death ratio to balance the classes, though that would also be a pretty bad way to do it.

except that 9/10 times game metrics are not representative of what is actually going on.

 

metrics usually take into account the whole population. which results in comparing samples with different sizes, which you generally shouldnt do.

 

it also means that the metrics are heavily diluted the greater the population. bioware should be using the top 5% of each class and adjusting sample sizes to make sure all ACs are compared equally.

 

you cannot objectively compare 5,000 Maras to 500 Mercs, and balancing them game on the performance of bad players is a terrible idea

While I detest your idea that arsenal mercs should get an instant tracer missile, you're pretty much killing it in this thread. Great posts all, at least through page 3.

DPS Operative: < 1%

 

DPS Mercenary: < 1%

 

DPS Sorcerer: 3%

Healing Operative: > 99%

 

Carnage Marauder: > 99%

 

Pyrotech Powertech: 92%

 

Shieldtech Powertech: 4%

 

You get the idea...

I don't think I;ve actually seen a rated team without one of those. Winning team or not.

You understand that just the fact that they used that as an example goes to show how clueless they can be about PvP and the metrics... right?

 

I'm not going to even go into the many problems about creating "balance" in MMO PvP... but it surely can NOT be done with mathematics.

 

They need to either play the game... of listen to an intelligent player base or G T F O.

You'd be surprised what can be done with mathematics. We're basically wizards.

Not that I'm defending the devs cause I don't think they could balance PVP if their lives quite literally depended on it, but of the 8 ACs, 7 of them have a rated viable spec, and I'd say their massive overbalance of classes via nerfs in 1.2 had as much to do with player QQ as it did metrics.

Again, not saying they're doing it right, but mathematics properly applied could help a ton with balancing. Most forum posters are dumb anyway. Wouldn't trust them to balance any more than I trust the devs.

the devs have acknowledged PvP as a majority playstyle for a while. back in Feb or March a dev said that >50% of players participated in wzs, while ~35% participated in endgame Ops. why they have not made more effort to improve PvP, which clearly a large part of their playerbase enjoys, is beyond me

Yeah this is one thing that I simply cannot wrap my head around. I remember them saying that. Lets be honest, PvP has been on the short end of everything that this game does and especially since that statement was released. One warzone in a year, with another coming soon™ but with all the different gear grinds and class balance being on a roller coaster I just lose a little faith every day.

In regards to the point about the only reason we complain about smashers is because there are alot of them. I remember reading in a previous statement by the devs that they would see if people are playing the spec (Rage/Focus) alot and fix/adjust it accordingly. And now they are saying nothing more about it or changes to the spec? Wow.

As long as teams of lolsmashers aren't roaming the warzones in hordes stunlocking people until they unsub, there's nothing to see here, folks.

Some players are never going to be happy, respectful, considerate or patient. They blow things out of proportion, deal only with absolutes and will never say a positive thing about anything or anyone unless it agrees completely with their own position. These players, many of the posters above, are good at venting and that is all.

 

I'll take BW's analytics over most of the posters on this board any day of the week. But I'll take numbers I can gather/monitor/verify and analyze myself over anyone's at BW. The main problem is that I/we have access to very few of those numbers. And another big problem is that many people have opinions they will defend to the death based on no verifiable facts (opinions like: "the world is flat" and "ur is the best defensive ability in the game" and "sorcs suck at team play" and "maras are overrunning the game" - even though there are far more sorcs than anything else in the 21-50 range and @50 on both POT5 and Jung Ma last time I checked).

Analysis of cold hard statistical data can be incredibly valuable if the models are applicable, meaningful and reflect reality. Right now however, they appear to be saying that the sky is pink and trees are made of cotton candy - and nobody has thought to poke their head out of a window and take a look.

The buffs to focus/rage were a big eye-opener that all was not well with their use of data.

Analysis of cold hard statistical data can be incredibly valuable if the models are applicable, meaningful and reflect reality. Right now however, they appear to be saying that the sky is pink and trees are made of cotton candy - and nobody has thought to poke their head out of a window and take a look.

 

The buffs to focus/rage were a big eye-opener that all was not well with their use of data.

oooh! I just saw WRECK-IT RALPH! too!

Everyone, it's important to remember that the answers Jonathan gave are just a few examples of metrics our analytics team looks at for certain specific purposes. Class balancing overall is much more complicated than looking at a couple stats - these were just a few examples of things the analytics team looks at and it wasn't intended as a comprehensive deep-dive into PvP balancing. It also wasn't a statement that we feel everything is completely perfect. We're always looking at player feedback, testing, and much more to to adjust the classes, and will continue to do so.

Could you please tell us if anything will be done class balance wise for patch 1.6? As its the patch with a new WZ in it would be the ideal time to do such a thing.

Could you please tell us if anything will be done class balance wise for patch 1.6? As its the patch with a new WZ in it would be the ideal time to do such a thing.

I'd really like to know the answer to this as well.

Everyone, it's important to remember that the answers Jonathan gave are just a few examples of metrics our analytics team looks at for certain specific purposes. Class balancing overall is much more complicated than looking at a couple stats - these were just a few examples of things the analytics team looks at and it wasn't intended as a comprehensive deep-dive into PvP balancing. It also wasn't a statement that we feel everything is completely perfect. We're always looking at player feedback, testing, and much more to to adjust the classes, and will continue to do so.

Is this why you ignore all class balance feedback and buff the alredy most powerful classes/AC ?

Allison,

The problem is not that the majority of us think that you are utilizing KDR to create PVP-balance.

The issue is that your PVP community is feeling a dearth of genuine engagement from the Developers.

Jonathon's choice of mentioning KDR is problematic. First, I doubt a majority of the posters here accept that statement as factual, for the myriad reasons already addressed. Secondarily, it speaks directly to the concern the PVP Community has over balance when a qualitatively poor metric is given as Developer Proof of Balance.

A quick review of the PVP forums indicates three major stains of concern:

1) Marauder Rage Spec – Is the Smash "truly" working as intended, and its effect on balance

2) Sorcerer Static Barrier – Clearly a failed attempt to balance a weakened spec

3) DPS Mercs/Commandos – Nigh useless

Developer engagement with these issues has been silence on the first point after an initial "we are monitoring it" stage, an ambiguous recognition of the broken mechanic on the second point, and little more than a "we know its utterly broken and may be addressed in the future" reply to the Mercs.

Perhaps an increased Developer-Community engagement effort, which makes it clear to the community where BW thinks Balance is in PVP at the moment, and more concrete statements (which would require concomitant decisions about balance on BW's part) to the concerned portions of the community.

Yes, that will open you up to considerable criticism.

However, I would think that maintaining a clear and open dialogue with the otherwise dwindling community should be prioritized over the headaches of the already extant trolls and QQers.

Yes, that will open you up to considerable criticism.

However, I would think that maintaining a clear and open dialogue with the otherwise dwindling community should be prioritized over the headaches of the already extant trolls and QQers.

QFT.

I know LoL is a pvp game only but just look at the amazing communication they have around their game and its balance. I know it hasnt always been the case but gosh it feels so good to go on riot's website and actually know what will happen.

Allison,

 

The problem is not that the majority of us think that you are utilizing KDR to create PVP-balance.

 

The issue is that your PVP community is feeling a dearth of genuine engagement from the Developers.

 

Jonathon's choice of mentioning KDR is problematic. First, I doubt a majority of the posters here accept that statement as factual, for the myriad reasons already addressed. Secondarily, it speaks directly to the concern the PVP Community has over balance when a qualitatively poor metric is given as Developer Proof of Balance.

 

A quick review of the PVP forums indicates three major stains of concern:

1) Marauder Rage Spec – Is the Smash "truly" working as intended, and its effect on balance

2) Sorcerer Static Barrier – Clearly a failed attempt to balance a weakened spec

3) DPS Mercs/Commandos – Nigh useless

 

Developer engagement with these issues has been silence on the first point after an initial "we are monitoring it" stage, an ambiguous recognition of the broken mechanic on the second point, and little more than a "we know its utterly broken and may be addressed in the future" reply to the Mercs.

 

Perhaps an increased Developer-Community engagement effort, which makes it clear to the community where BW thinks Balance is in PVP at the moment, and more concrete statements (which would require concomitant decisions about balance on BW's part) to the concerned portions of the community.

 

Yes, that will open you up to considerable criticism.

However, I would think that maintaining a clear and open dialogue with the otherwise dwindling community should be prioritized over the headaches of the already extant trolls and QQers.

I'd really love to see this actually. I hope the next Q&A or next State of the Game Discussion focuses entirely on PvP. I'd wager a guess that PvP remains one of the most popular components of the game -- I think it's time to throw the PvP crowd a bone and discuss some of the changes you've got in store. Ancieny Hypergates and Elite War Hero in 1.6 is really the first content the PvP has gotten since 1.2 in April (War Hero, new Expertise curve, Novare Coast, ton of class balanaces). We also know that you've got something cooking for Ilum for the anniversary of 1.1 and it seems, at least based on the patch notes thus far, the soonest we'll see class changes is in 1.7 (which will be January/Feburary, presumably).

IceHawk - thank you for posting your feedback in such a constructive way. I don't have answers to all of your concerns right now, but I want you to know the community team is always working to escalate concerns and issues to the developers and get responses when we can (when we're sure of a course of action, have feedback requests, or have any updates to an issue). I mostly wanted to call out your post as a really model post for constructive feedback. We'll definitely continue working to bring more answers to questions and concerns.

 

Also - thanks for the suggestions on a State of the Game or Q&A dedicated to PvP. I can't promise anything, but will bring up the feedback!

While I can only speak for myself, all I want is little updates on things that are in the works or any future plans you have towards PvP. I think the main reason I kept my sub was to check out this new warzone. Its the little things that keep players going.