View Single Post

Tibbel's Avatar

02.02.2012 , 06:52 PM | #9
Thanks for this testing!

Quote: Originally Posted by QisaBluevixen View Post
Is this enough to be statistically valid?
Well, since you asked...

This test is an attribute test, which means each data point is either a TRUE or a FALSE. Let's set our null hypothesis to say: "The proc rate is 25%."

The null hypothesis is the claim we are trying to test. We can never confirm it through testing, only reject it or fail to reject it with a certain confidence. Confidence is typically denoted as γ or as (1 - α) and is a value between 0 and 1.

The resulting data from the test leads to a p-value, which is the chance that the actual conditions are at least as extreme as the observed. (In this case, it's the chance that you would get the test results you did if the proc rate is actually 25%.) Like confidence, the p-value is also a number between 0 and 1. It's dependent on the hypothesis, the observed data, and the sample size. (Look up the math if you care about it -- I used Minitab.)

If p < α, then we reject the null hyposthesis (i.e. the results are statistically significant); otherwise, we fail to reject it.

So, for your first test, your null hypothesis is: "The proc rate of 2/2 CM is 25%."
Let's say we want to be 95% confident, so α = 0.05.

14 ticks out of 153 trials results in a p-value of: p < 0.001. Since p < α, we can reject the null hypothesis with a confidence of 95%.

In other words: YES, your test results are enough for us to say the proc rate is not 25%.


You also conducted a second test, which tested the null hypothesis: "The proc rate is the same, whether the target is level 49 or level 50."
Let's shoot for the same 95% confidence target, so α = 0.05.

Using your data...
Lv 50: 8 procs in 108 opportunities
Lv 49: 6 procs in 45 opportunities

The resulting p-value is: p = 0.355. Since p > α, we fail to reject the null hypothesis.

In other words, NO, we cannot say from these results that the proc rate is different for level 49 vs level 50.

Hope that helps.

Oh, by the way:
Quote: Originally Posted by QisaBluevixen View Post
LFL in Alderaan Places
This has been my personal guild status note since launch.
You know... they say great minds think alike.