As I've been reading up on Lesswrong.com, it's occurred to me that I really do not know much about statistics/probability at all. I never took a college level statistics course. I currently do now have time to do so. But I have 3 hours of train-commute every day and could use something to fill it with. So is there a book (I guess basically a math textbook) that people recommend that's well written and recommendable as a way to study statistics/probability on your own?
(I'm not sure if there's a particular difference between saying "studying statistics" and "studying probability.")
[ October 20, 2010, 01:19 PM: Message edited by: Raymond Arnold ]
Posted by fugu13 (Member # 2859) on :
Get a copy of the Cartoon Guide to Statistics. No, I'm not joking. It will take you handily through content the equivalent of a first semester college statistics course for non-math majors, with probably a good deal better absorption and practical understanding than such a course.
I have never seen a better introductory statistics book than the cartoon guide.
There is a difference between statistics and probability in math terms. Statistics is about data. Probability is about random variables. They're closely linked, of course, but very differently studied. A textbook in probability would not be very useful to you, but a textbook in statistics would be.
edit: afterwords, a good popular book to address some common situations and ways of approaching real life numerical mistakes is Paulos' Innumeracy.
Posted by rivka (Member # 4859) on :
It's not precisely what you're looking for, but it's close enough I figured I'd mention it. I have a copy of this book and as it's just gathering dust on the shelf, it's yours if you want it enough to pay for shipping (through Goodreads' bookswap, since I'm all set up for that already).
Posted by rivka (Member # 4859) on :
quote:Originally posted by fugu13: Get a copy of the Cartoon Guide to Statistics.
I heartily endorse not only this book but the series. It's awesome.
Posted by fugu13 (Member # 2859) on :
Did the description writer mean to write "experiential design", or is that a typo for "experimental design"?
Posted by rivka (Member # 4859) on :
Book's at home; I'm at work. Amazon's look-inside to the rescue!
Typo.
Posted by Destineer (Member # 821) on :
There's actually some good introductory work on this topic by philosophers.
I've gone ahead and ordered The Cartoon Guide to Physics for my oldest. Thanks for the recommendation!
Posted by fugu13 (Member # 2859) on :
Hacking's book does look interesting, and I've seen Andrew Gelman recommend another of his books, so it seems he's one of the philosophers who doesn't make a complete muck of modern Bayesian practice.
*recalls fond memories of the cartoon guide to physics*
If you're looking for educational texts, the cartoon guide to genetics is also worth checking out.
Posted by rivka (Member # 4859) on :
I like the chemistry one too.
Posted by fugu13 (Member # 2859) on :
Ooh, I don't think I've read that one. I'll have to find a copy.
Posted by Lisa (Member # 8384) on :
Go to a casino. It's probably the quickest and most painful way to learn basic probability.
Posted by rivka (Member # 4859) on :
quote:Originally posted by fugu13: Ooh, I don't think I've read that one. I'll have to find a copy.
Visit L.A., and you can borrow mine.
Posted by mr_porteiro_head (Member # 4644) on :
quote:Originally posted by Lisa: Go to a casino. It's probably the quickest and most painful way to learn basic probability.
Patently false, considering the fact that they stay in business. Posted by King of Men (Member # 6684) on :
So do universities, yet you would not argue on those grounds that they teach nothing. Posted by fugu13 (Member # 2859) on :
Unsurprisingly, the study of modern statistics was born from the investigations of a gambler (this is covered in the cartoon guide ).
Posted by Sa'eed (Member # 12368) on :
I've taken an upper division course in probability (not with the stat department but with the math department.) The first half of the class was fun. Then calculus got involved and there was no more fun.
Posted by rivka (Member # 4859) on :
How do you get through even half of an upper division probability class without calculus?
Posted by fugu13 (Member # 2859) on :
At least at IU, every prob/stats course the math department had was listed at 300 level or above, including the introductory for non-math majors that tried to avoid actual math as much as possible.
Posted by rivka (Member # 4859) on :
Weird.
Posted by Sa'eed (Member # 12368) on :
We used this book. We covered 8 of the 10 chapters, and the first 4 are:
Combinatorial Analysis Axioms of Probability Conditional Probability Random Variables
Then it was spring break. After the break, we started chapter 5, which was Continuous Variables and which involved calculus.
Posted by Destineer (Member # 821) on :
quote:Originally posted by fugu13: Hacking's book does look interesting, and I've seen Andrew Gelman recommend another of his books, so it seems he's one of the philosophers who doesn't make a complete muck of modern Bayesian practice.
Huh, I was just looking at Gelman's complaints about subjective Bayesian epistemology. I find them a bit puzzling. He observes that Bayesian methods are often used in science for model-fitting projects that don't fit the description of inductive reasoning. So far so good. Then he wants to say that this is a problem for the view that Bayesian conditionalization describes inductive reasoning.
I don't get why this last bit is supposed to follow. Couldn't Bayesian updating work both as a model-fitting method, and also as an accurate description of inductive belief-change?
If his sole opponent is the philosopher of science who thinks that all scientific activity is constituted by inductive belief-change describable in a Bayesian way, he'd have a good argument. But his target seems to be broader than that.
Posted by fugu13 (Member # 2859) on :
He is a bit confused over it, I think because he doesn't approach it from a philosophical direction at all. To him, Bayesian approaches, as actually applied by working statisticians, are a tool that can be applied in objective ways to obtain better objective results.
He's not so concerned with Bayesianism as it exists in philosophy; he wants to say that almost every time a philosopher says what Bayesianism "is", they're making a big mistake about the place you'll see Bayesian approaches most explicitly used: statistics.
He also doesn't think Bayesianism well describes the scientific method, because of his strong beliefs in models, and model checking, and that the models are even so just useful tools. That is, his views on models are further from "more correct, less correct" that would be appropriate for an inductive belief-change story, and nearer to "more useful, less useful", which has less to do with belief change and more to do with objective measurement of results. Advances in his field come about least because of theories gaining traction from repeated experiment, and most because of advances in statistical theory and computational capability.
But the second bit is more confused, in part because I don't think he thinks about it as much as the first bit.
Personally, I'm very skeptical of a straightforward inductive belief-change story, myself, because that doesn't seem to be how scientists proceed. There's definitely a Kuhnian component, where new ideas "overthrow" old ideas, of course because they're more compelling given evidence, but also because the situation is ready for them to be more compelling due to culture of science reasons. The idea that scientists are assimilating evidence and updating their priors until one theory is preferred to another theory is pretty, but too pretty, and doesn't seem to match the fairly abrupt changes well, where an entire camp will split off from the main group at once -- are there priors really all so similar? I'm sure there are philosophers who have done work on reconciling the basic idea with the messiness of it all, but I sadly haven't read enough to know of them.
Posted by fugu13 (Member # 2859) on :
As for why I use him as a vetting tool, if a philosopher is writing about statistics, and I want to learn how to do practical statistics from that writing, I want a treatment of Bayesianism as it applies to statistics.
Posted by Destineer (Member # 821) on :
quote:Originally posted by fugu13: He is a bit confused over it, I think because he doesn't approach it from a philosophical direction at all. To him, Bayesian approaches, as actually applied by working statisticians, are a tool that can be applied in objective ways to obtain better objective results.
He's not so concerned with Bayesianism as it exists in philosophy; he wants to say that almost every time a philosopher says what Bayesianism "is", they're making a big mistake about the place you'll see Bayesian approaches most explicitly used: statistics.
He also doesn't think Bayesianism well describes the scientific method, because of his strong beliefs in models, and model checking, and that the models are even so just useful tools. That is, his views on models are further from "more correct, less correct" that would be appropriate for an inductive belief-change story, and nearer to "more useful, less useful", which has less to do with belief change and more to do with objective measurement of results. Advances in his field come about least because of theories gaining traction from repeated experiment, and most because of advances in statistical theory and computational capability.
But the second bit is more confused, in part because I don't think he thinks about it as much as the first bit.
Personally, I'm very skeptical of a straightforward inductive belief-change story, myself, because that doesn't seem to be how scientists proceed. There's definitely a Kuhnian component, where new ideas "overthrow" old ideas, of course because they're more compelling given evidence, but also because the situation is ready for them to be more compelling due to culture of science reasons. The idea that scientists are assimilating evidence and updating their priors until one theory is preferred to another theory is pretty, but too pretty, and doesn't seem to match the fairly abrupt changes well, where an entire camp will split off from the main group at once -- are there priors really all so similar? I'm sure there are philosophers who have done work on reconciling the basic idea with the messiness of it all, but I sadly haven't read enough to know of them.
There's work on fitting scientific changes into a Bayesian framework, but it's pretty unimpressive compared with the scale of the problem. You're right to think that this is a very serious issue for Bayesian updating as a theory of scientific rationality.
Posted by katharina (Member # 827) on :
Try Innumeracy. It goes over the most common mistakes people make when thinking about probability.
Posted by fugu13 (Member # 2859) on :
Indeed. That's why I recommended it in the second post in this thread Posted by katharina (Member # 827) on :
By the mouths of two or three witnesses shall every truth be established.
-OR-
The superior post has a link. Posted by SenojRetep (Member # 8614) on :
quote:Originally posted by Destineer: There's work on fitting scientific changes into a Bayesian framework, but it's pretty unimpressive compared with the scale of the problem. You're right to think that this is a very serious issue for Bayesian updating as a theory of scientific rationality.
Is there a micro/macro divide within the use of Bayesian analysis in the scientific community? It seems like at least some scientists use Bayesian analysis techniques for individual analysis, but maybe there are no macro level applications of Bayesian analysis to broad adoption of new scientific models/theories? Or am I misunderstanding your statement?
Posted by fugu13 (Member # 2859) on :
quote:Is there a micro/macro divide within the use of Bayesian analysis in the scientific community? It seems like at least some scientists use Bayesian analysis techniques for individual analysis, but maybe there are no macro level applications of Bayesian analysis to broad adoption of new scientific models/theories? Or am I misunderstanding your statement?
Even if you buy into the Bayesian updating story for how science works, it isn't a conscious thing. It is a description of an underlying mental process that some philosophers view as compelling (or at least possible). There aren't any scientists (note: there might be one or two. If there are, they're probably crazy) that come up with gigantic distributional models that encompass multiple scientific theories, perform experiments that allow them to update those models, then take a look and see which theory is currently winning, probability-wise.
Personally, I think it would be a really stupid way to do science, for a simple reason: setting up the model would involve way, way too much implicit bias.
I'm not completely sure what you mean by "individual analysis" and "macro level applications", but Bayesian models are the centerpiece of modern statistical modeling and machine learning. They're used all the time on diverse problems and diverse datasets.
Posted by SenojRetep (Member # 8614) on :
Right. The fact that they're used all the time was my point.
Say we want to predict the upcoming election, based on a set of observables (like poll results, for instance). I could imagine constructing a likelihood model using data from previous elections, and use Bayesian analysis to sequentially update a posterior given all the polls in this election to date. This is a social science example of what I meant by a "micro" application: a clearly defined problem for which constructing likelihood models is reasonable.
What I meant by "macro" models is more like scientific adoption of new theories. I don't imagine that each individual in the community creates likelihood functions of how likely observables are given the old theory vs. the new theory. Rather, they're convinced over time by some combination of compelling story-lines constructed in presentations, posters and papers and social pressure such that eventually a new explanation is adopted. They could go to the trouble of generating those likelihood functions, push the data through, and select some sort of maximum a posteriori estimate of the variable under consideration, but I don't imagine that's how it's done (even on a "black box" cognitive level).
Personally I imagine "macro" hypothesis adoption is probably better modeled by a contagion model, where social networks play a significant role in the adoption (or not) of new theories, new "hot" topics of research, etc.
Posted by fugu13 (Member # 2859) on :
quote:What I meant by "macro" models is more like scientific adoption of new theories. I don't imagine that each individual in the community creates likelihood functions of how likely observables are given the old theory vs. the new theory. Rather, they're convinced over time by some combination of compelling story-lines constructed in presentations, posters and papers and social pressure such that eventually a new explanation is adopted. They could go to the trouble of generating those likelihood functions, push the data through, and select some sort of maximum a posteriori estimate of the variable under consideration, but I don't imagine that's how it's done (even on a "black box" cognitive level).
Then I'm confused what you mean by an "application". That generally implies some sort of conscious, deliberate usage, not just a philosophical model of a mental process.
Posted by SenojRetep (Member # 8614) on :
I guess I had in mind something akin to the historical adoption of a relativistic explanation for planetary orbits over the older, Newtonian model. As I (perhaps incorrectly) recall, Einstein's model accounted for observations regarding Mercury's orbit that the Newtonian model could not, which led to its broad adoption as the model du jour.
So, if I were to envisiage a "macro" application of Bayesian belief revision here, I would ask "what is the probability of observing <planetary orbital path 1-9> given the Newton model" and "what is the probability of observing <planetary orbital path 1-9> given the Einstein model." Then I'd put some priors on the two models (probably uniform) and see which had the larger posterior likelihood.
If all scientists did this independently, they would likely come up with a variety of likelihood functions, each resulting in a different posterior probability over the two models. However consensus could still be reached through a couple of methods: 1) revision of likelihood models based on dialogue (thereby exposing individuals to evidence they hadn't previously accounted for) or 2) further observation of orbital paths. In the end, even if the likelihood models didn't converge (people's subjective assessments probably wouldn't), some consensus on the posterior maximizing hypothesis probably would.
Now, I'm not saying that such a process would be "right" or "good" or anything. In fact I think such a process would be a pretty inefficient way of going about things. However, I don't think the "micro" application I suggested before (or others like it) is similarly problematic. Which led me to posit that maybe there is a micro/macro divide in the sorts of applications to which scientists (speaking broadly) are likely to use Bayesian analysis.
![[Smile]](smile.gif)
![[Big Grin]](biggrin.gif)
).
![[Razz]](tongue.gif)