When we make changes to the mean, standard deviation, and number of samples within the applet, what are some some things we should be noticing?
As we play around with the values in this applet, we should be noticing at least three things:
- As we increase the number of samples, the two sample distributions begin to resemble a normal distribution.
- As the means of the two distributions differ by “small” amounts, the p-value remains very small (less that 0.05) in most cases. As we see in the applet, this tells us that the distributions are “significantly different”.
- Finally, we can notice that by making the means equal and making the standard deviations somewhat different (1 and 2 for example) or wildly different (1 and 100 for example), the p-value remains “large” (greater than 0.05) in most cases. Again, the applet tells us that this means that the distributions are not “significantly different”.
These observations provide experimental hints about how the mean and standard deviation weigh on the similarity of two distributions. Play around with the values in the applet to understand how changes in these values change the p-values to give concrete meanings to “small” and “large” in the points above.