Comments on Baseball, T-tests and statistical surprisesTypePad2011-03-30T16:56:48ZBlog Administratorhttp://blog.revolutionanalytics.com/tag:typepad.com,2003:http://blog.revolutionanalytics.com/2011/03/baseball-t-tests-and-surprises/comments/atom.xml/Andrew Clark commented on 'Baseball, T-tests and statistical surprises'tag:typepad.com,2003:6a010534b1db25970b01543295480f970c2011-05-27T12:32:38Z2011-05-27T12:32:38ZAndrew Clarkhttp://www.premiersoccerstats.comI redid the work excluding all players with less than 100 Plate Appearances. This niftily excluded all pitchers - as...<p>I redid the work excluding all players with less than 100 Plate Appearances. This niftily excluded all pitchers - as the maximum PA for any pitcher in either season was in the 90s - and will also exclude late season callups who often hit worse than MLB regulars</p>
<p>The t-test gave the following result</p>
<p>t = 0.4475, df = 822, p-value = 0.6546<br />
alternative hypothesis: true difference in means is not equal to 0 <br />
95 percent confidence interval:<br />
-0.003606420 0.005736718 <br />
sample estimates:<br />
mean in group 1990 mean in group 2010 <br />
0.2566466 0.2555814 </p>
<p>The means could hardly be any closer. </p>
<p>The density plots are much closer to a normal distribution. Fielding position is probably also a factor as catchers, for instance, are not expected to have as high a batting average as those in less crucial areas of the diamond<br />
</p>Barry commented on 'Baseball, T-tests and statistical surprises'tag:typepad.com,2003:6a010534b1db25970b014e872bb03d970d2011-04-01T19:46:22Z2011-04-01T19:46:22ZBarryI think the part-time players ("outliers" if you wish) probably impact the variability as well... In 1990, there were 26...<p>I think the part-time players ("outliers" if you wish) probably impact the variability as well... In 1990, there were 26 teams with a total of 650 roster spots (25 per team), but there were 740 players in the list (many, but not all, of the extra 90 are the part-timers that are probably disproportionately near the extremes)... In 2010, there were 30 teams and 750 roster spots, but the 2010 data shows 944 players... That's roughly 100 extra regulars, but also 100 more part-timers... I wouldn't be surprised that removing the part-timers would also likely remove most of the variability differences...</p>Wade commented on 'Baseball, T-tests and statistical surprises'tag:typepad.com,2003:6a010534b1db25970b014e872a805e970d2011-04-01T15:29:32Z2011-04-01T15:29:32ZWadeWhat about the Central Limit Theorem? Even if trimodal, the sampling distribution of the mean will be normally distributed with...<p>What about the Central Limit Theorem? Even if trimodal, the sampling distribution of the mean will be normally distributed with 700+ samples. The assumption of normality( or more accurately a t-dist) is on that distribution, not the parent distribution. </p>
<p>Here is a crazy mixture of normal, lognormal, gamma, and beta that is similar to what was shown for the actual data. Look at the sampling distribution (takes about 1 min to run):</p>
<p></p>
<p>xpts<-seq(0,1,.01)</p>
<p>par(mfrow=c(3,1))<br />
plot(y=<br />
0.4*dnorm(x=xpts,mean=0.3,sd=0.035)<br />
+0.4*dlnorm(xpts,meanlog=log(0.265),sdlog=0.3)<br />
+0.15*dgamma(xpts,shape=0.2,scale=0.5)<br />
+0.05*dbeta(0.9*xpts+0.01,shape1=0.5,shape2=0.5),<br />
x=xpts,type="l",xlim=c(0,1),main="Mixture Density",ylab="f(x)",xlab="x")</p>
<p></p>
<p>plot(density(<br />
c(<br />
rnorm(400,mean=0.3,sd=0.035),<br />
rlnorm(400,meanlog=log(0.265),sdlog=0.3),<br />
rgamma(150,shape=0.2,scale=0.05),<br />
rbeta(50,shape1=0.5,shape2=0.5)<br />
))<br />
,xlim=c(-0.1,1.1), main="Typical Kernel Density Estimate")</p>
<p><br />
xbar<-c(1:5e4)</p>
<p>for(i in 1:5e4){<br />
xbar[i]<-mean(c(<br />
rnorm(0.4*740,mean=0.3,sd=0.035),<br />
rlnorm(0.4*740,meanlog=log(0.265),sdlog=0.3),<br />
rgamma(0.15*740,shape=0.2,scale=0.05),<br />
rbeta(0.05*740,shape1=0.5,shape2=0.5)<br />
)<br />
)<br />
}</p>
<p>xpts2<-seq(0.240,0.270,0.0001)<br />
plot(density(xbar), main="Sampling Dist of Mean w/Normal Overlay (red)")<br />
lines(y=dnorm(x=xpts2,mean=mean(xbar),sd=sd(xbar)),x=xpts2,<br />
type="l",col="red",lwd=2,lty=2)</p>
<p><br />
Looks pretty good to me. Now, one could also debate that we have census rather than a sample here but then that takes the fun out of baseball data altogether!</p>
<p>Also, the Wilcoxon-Mann-Whitney isn't just a test of location, it will detect differences in shape/variability (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1120984/)</p>
<p>So that the fact that it finds a difference may just be due to differences in variability.</p>Robert commented on 'Baseball, T-tests and statistical surprises'tag:typepad.com,2003:6a010534b1db25970b014e604e2c5e970c2011-04-01T10:26:33Z2011-04-01T10:26:33ZRobertAuthor probably forget to take into account an assumption which is crucial to use Wilcoxon Rank Sum Test. I mean...<p>Author probably forget to take into account an assumption which is crucial to use Wilcoxon Rank Sum Test. I mean symmetric distribution. Obviously, none of those two are symmetric therefore the test results are questionable. </p>Joseph Rickert commented on 'Baseball, T-tests and statistical surprises'tag:typepad.com,2003:6a010534b1db25970b014e604b7585970c2011-03-31T23:21:59Z2011-03-31T23:21:59ZJoseph RickertThank you all for the comments. It is nice to just do some basic statistics. I should have thought to...<p>Thank you all for the comments. It is nice to just do some basic statistics. I should have thought to adjust the axis for the kernel plots, sorry for the confusion. Doing permutation tests and the trying the Welch-Satterthwaite adjustment sound like good ideas. However, I am not so sure about removing the outliers first thing. Doing that seems to me to shift the question to something like: how different are the "real" hitters?</p>Millsy commented on 'Baseball, T-tests and statistical surprises'tag:typepad.com,2003:6a010534b1db25970b0147e3a5c5b1970b2011-03-31T19:35:27Z2011-03-31T19:35:27ZMillsyFirst thing I noticed here is that it's pretty obvious there are a lot of players with only a couple...<p>First thing I noticed here is that it's pretty obvious there are a lot of players with only a couple at bats in the data set. Batting average has a lot of noise even for a guy with a full season of at bats.</p>
<p>If you set a minimum of at bats for a season, you'll find the distribution to have lots of players bunched up on the left, with a long right tail.</p>
<p>Honestly, the AB filter should be something in the realm of no less than 300. It's important to of course check assumptions, but even before that it's important to use common sense.</p>John Myles White commented on 'Baseball, T-tests and statistical surprises'tag:typepad.com,2003:6a010534b1db25970b014e604a4639970c2011-03-31T18:44:25Z2011-03-31T18:44:25ZJohn Myles Whitehttp://johnmyleswhite.comWhy not run a permutation test?<p>Why not run a permutation test?</p>Rick Wicklin commented on 'Baseball, T-tests and statistical surprises'tag:typepad.com,2003:6a010534b1db25970b014e604a410d970c2011-03-31T18:40:03Z2011-03-31T18:40:03ZRick Wicklinhttp://blogs.sas.com/imlIt's always great to see some good old-fashioned data analysis! Thanks for the guest blog, Joseph. As the other comments...<p>It's always great to see some good old-fashioned data analysis! Thanks for the guest blog, Joseph.</p>
<p>As the other comments point out, you can restrict "at bats" to eliminate the extreme outliers (maybe use, AB > 20). Also, restrict the kernel density to only consider AVG>=0. (I don't know how to restrict the KDE in R off the top of my head, although I know how to do it in another language.)</p>
<p>Lastly, the standard t-test is fairly robust to non-normal data, but it can be sensitive to unequal variance, which I'm guessing will still be present even after you filter by AB>20. In that case, you might want to use the Welch-Satterthwaite adjustment to the t-test, which I think has better power (???)than the more-general Mann-Whitney test.</p>Jared commented on 'Baseball, T-tests and statistical surprises'tag:typepad.com,2003:6a010534b1db25970b014e604a099e970c2011-03-31T17:56:11Z2011-03-31T17:56:11ZJared@Bill Svec That's an artifact from the kernel density estimate; it doesn't know the range of the data unless you...<p>@Bill Svec That's an artifact from the kernel density estimate; it doesn't know the range of the data unless you supply it. If you look at the rug (the green lines) you'll see that the data stops at 0 and 1 as it should.</p>Jon Peltier commented on 'Baseball, T-tests and statistical surprises'tag:typepad.com,2003:6a010534b1db25970b0147e3a551c3970b2011-03-31T17:56:05Z2011-03-31T17:56:05ZJon Peltierhttp://peltiertech.com/WordPress/I was going to suggest excluding batters with less than a certain number of at-bats. Obviously those at around .400...<p>I was going to suggest excluding batters with less than a certain number of at-bats. Obviously those at around .400 and higher would be removed. So would much of the peak which is strangely drawn centered on .000.</p>
<p>I'm a little surprised by the size of this peak. Are there that many pitchers and players who got only a handful of at-bats? Maybe each player's contribution should be weighted by number of plate appearances. I'm sure without even filtering out batters with few at bats, the peak at zero would then disappear.</p>Mike commented on 'Baseball, T-tests and statistical surprises'tag:typepad.com,2003:6a010534b1db25970b014e8724dfe4970d2011-03-31T17:36:34Z2011-03-31T17:36:34ZMikeIf you look at the density plots, you notice that a number of batters had a batting average of zero...<p>If you look at the density plots, you notice that a number of batters had a batting average of zero (no hits) and that some had a batting average of 1.0 (100% hits!). Those guys with a 100% batting average must be billionaires by now. You would never lose with them in your lineup! Seriously, a godd batting average is around 0.3. Batters at the extremes of the distribution likely have a low number of attempts. Since it is reasonable to say we are only interested in the performance of typical batters in our comparison, it would be logical to apply a filter to the data based on the number of hits. This would likely remove the extreme values from the distribution. Whether this would affect the results of your statistical tests is an open question. </p>Bill Svec commented on 'Baseball, T-tests and statistical surprises'tag:typepad.com,2003:6a010534b1db25970b014e8724c108970d2011-03-31T17:09:49Z2011-03-31T17:09:49ZBill SvecHopefully this isn't a dumb question -- I'm new to all of this, but in your density plots above the...<p>Hopefully this isn't a dumb question -- I'm new to all of this, but in your density plots above the curve appears to extend below 0.0 a little bit. How could a player have a negative batting average? Or am I interpreting this incorrectly?</p>
<p>Thanks</p>