Studying Statistics: Part II

  Chapter 3 of The Signal and The Noise (you know what, I'm just gonna call it TSATN for brevity from now on!) offered an overview of the impact that statistical models have had on baseball analysis. Having read Moneyball in 2003, I was aware of the sabermetric movement and how obscure stats were being used to find value in players where traditional stats don't. I was also a huge fantasy baseball nerd in the early 2000s and was interested to find that Nate Silver had put together the PECOTA projections that I had relied on to win a few leagues in 2004. 

  This chapter centered around some anecdotes from Silver's interviews (or attempts to interview) with Dustin Pedroia, Bill James and Billy Beane. It explored what made the Moneyball philosophy work (considering a lot of information) and why it doesn't work so well today (everybody is a fox). An illustration of this was given by the fact that pure stats based rankings like PECOTA performed better than scouts at it's advent; but now that scouting departments use hybrid models (considering much more information), they are the ones making better predictions. 

  Lesson 10 of Udacity's ST095 introduces t-distributions, degrees of freedom and one sample t-tests. When you have a small sample size and the standard deviation of the population is unknown, examining a t-distribution can help.

  This was an interesting lesson and I like that we got to work through an example of how to conduct a t-test. We first got our formulas in order for degrees of freedom, t-value, margin of error, standard error, confidence intervals, Cohen's d and $r^2$.  

  

  Then we were given some information on a population mean and a treatment. The assessment asked us to identify dependent variables, appropriate null and alternative hypotheses, and whether a one-tailed or two-tailed test should be used. By computing the values for our given formula, with the help of some additional information about sample standard deviation and mean, we were able to illustrate a t-distribution and place our test value on that distribution. This let us find a p-value (probability) for the test value $t$ so we can answer the questions of whether the results were statistically significant and meaningful. These two things have a different meaning in statistics than our conventionally understood definitions. Significance means that we have rejected the null hypothesis and the results are not likely due to chance. Meaningfulness depends on what was measured, the effect size, whether we can rule out random chance, and that there aren't any extraneous factors that might also explain the results. To finish up the lesson we computed our values for Cohen's d, an effect size that provides a magnitude of the relationship, the coefficient of determination, $r^2$, and found the margin of error for a 95% confidence interval.