After all of the noise that's been made about Underworld Trading missions being broken, biased or some other way twisted against what players need, I decided to do some data collection to see just how random UWT missions were. After 3000 measurements, I think there are some pretty interesting stories in the numbers. My main problem is that it's been about 15+ years since I've done some T-Tests and ANOVAs and the sorts of tests that need to be done to separate the "wheat from the chaff" here. So, if you have some expertise in the statistical methods mentioned and want to take a shot at examining UWT mission distributions, send me a PM. Like I said, the numbers have all been collected, they just need to be crunched.

Anyone ... anyone ... Buehler?

 

Would it help to entice any stats folks out there to say that there are some rather clear trends indicating that faction-based missions may be weighted to appear as much as twice as frequently and that with the uneven distribution of mission types across grades and between factional/non-factional, there most likely is a built-in (read: by design) bias for certain types of missions to appear, and certainly NOT the ones that people want the most?

 

As I read the numbers, we are being screwed by design ... I just need someone with a bit more expertise than me to make sure the assumptions I'm making add up and the tests I'm running are valid.

i've read your post 3 times. and i have no idea what you are talking about. UT isn't broken, people just want to get rare metals EVERY time and that's not how it works.

As I read the numbers, we are being screwed by design ... I just need someone with a bit more expertise than me to make sure the assumptions I'm making add up and the tests I'm running are valid.

 

Are you suggesting that we're getting screwed according to the types of missions that are available for players to run or by the returns on the missions?

i've read your post 3 times. and i have no idea what you are talking about. UT isn't broken, people just want to get rare metals EVERY time and that's not how it works.

 

I didn't say it was broken. Quite the contrary: just like with RE, I am certain that the system is working as designed. I did say that there is evidence of a bias in mission distributions that could only happen by BioWare programming it to work that way. The statistical tests that you have no idea about are the sorts of tools that could show just how biased things are. Yes, I realize that by saying "i have no idea what you are talking about" you may simply be disagreeing with me, but if you understood the stats involved you would know that you do not have enough information to make that decision. This is not a simple matter.

 

It may be because of all the complaints of RE being "broken" when it clearly isn't -- when RE is a very straight-forward application of simple probability of single, independent events -- that people who do understand that RE is working as designed have the same simple faith that the randomization schemes used in crew mission selection is also done using a "fair coin", a process of random selection that shows no "favoritism" to either gift or fabric or metal missions. What I am saying is that using a set of 3000 data points, not just the last few times I tried to run missions, I've been able to do a few simple statistical tests that demonstrate a bias in mission selection that cannot be explained by a random fluctuation and therefore indicate some systematic variation. I can also state that the probability that these tests have falsely indicated a pattern when there really wasn't one, it was just a highly improbably random fluke, is less than 0.001, or a chance of less than 0.1% of being wrong. Statistically speaking, when human decision-making is involved, that level of precision is quite high.

 

What I am also saying is that there are structural design decisions that BioWare made, such as the absolute availability of gift, fabric and metal missions by yield and by Grade, that even if BioWare was using a truly "fair coin" to select which five missions appear per Grade for each time your mission list gets randomized there would exist a bias in mission availability due to that structure. If you would take the time to look at the Darth Hater Database of Underworld Trading Missions, as I have, you would see quite clearly where the biases lay. I am not saying anything further, or more definitive, because as I said my knowledge of the more advanced statistical tests needed to look at these more complex issues has faded somewhat in the 18 years since I last used them, and I have enough respect for how misleading statistics can be when they are done properly yet twisted to indicate what they do not (the old "lies, damned lies and statistics" issue), let alone the damage they can do when not done properly in the first place, to keep my mouth shut about saying anything more.

 

So yes, I am putting teasers out there regarding things I can claim with a high degree of certainty to hopefully grab someone's attention who could work with me on this. To be blunt about it, the questions that BioWare's structuring of mission distributions raise involve a model that would require at least a Two-Way Analysis of Variance (ANOVA) test involving a Between-Subjects design with unequal population sizes and a sampling without replacement involving the whole populations of each group. The limited sizes of the populations and the fact that they vary by nearly a factor of 2 between grades is what raises all sorts of red flags for me in thinking about just "tossing" the numbers into Excel and running the tests anyway: they may violate some of the root assumptions that are required for valid ANOVA designs. The biggest problem overall, though, is the one distinct systematic bias that I have been able to verify and the interaction affects it might create. Because of these possible interactions, I cannot make any definitive claims based solely on BioWare's design and a logical argument assuming simple random assignment.

 

If you didn't understand my earlier posts, I am certain you didn't understand that. But it's not my intent to confuse people, rather, I am trying to make clear what stats knowledge is required so people with some stats knowledge can rule themselves out or not. This is not a situation parallel to whether "a 20% Chance for Success means I will always get a success in 5 tries or less" is right or wrong (that's patently wrong for anyone with an understanding of simple probability). But go ahead and look up some of those terms I tossed around in Wikipedia and you'll quickly find out how complex an issue this is.

Are you suggesting that we're getting screwed according to the types of missions that are available for players to run or by the returns on the missions?

 

What I can say definitively from the data I have is that your first point is correct. There is a systematic bias that makes certain missions more likely to be selected than other missions. Your second point is the issue I cannot answer because of the stats I no longer remember well enough. I cannot say yes or no to it, because the answer to that question is mixed up with what creates the first question you raised. In statistical testing, that is something known as an interaction effect. Detecting statistically meaningful interaction effects requires a bit of "finesse", something I can't quite provide. But because I know the interaction is there, I'm keeping my mouth shut about what the biases are. It's as if my car won't start because it seems that no gas is getting to the engine. I might think there is no gas in the tank, but it could also be that the fuel pump is broken or the fuel line is clogged. Rather than saying the right thing for the wrong reason, I'd rather wait until all the reasons are understood. Speaking out right now could just make a bigger mess of things.

if you don't like the 5 choices you have, just enter your ship and they change....don't like those? exit the ship and they change again. at least that's what i do, and i only run Rich missions most of the time

if you don't like the 5 choices you have, just enter your ship and they change....don't like those? exit the ship and they change again. at least that's what i do, and i only run Rich missions most of the time

 

But there are two things you are missing out on. Before you step in your ship and randomize your list again, should you choose missions out of the undesirable ones that are available, and does it matter which missions you choose? The answer is yes to both.