Maximum Likelihood Estimation (MLE)
Of the many statistical estimation techniques (yeah, in the end, it is just a technique, the whole solution is still can be a very bad approximation practically), MLE or maximum likelihood estimation is a popular one. Not having to go through the technicalities, what got me thinking on this at a qualitative level is a talk I was listening to (somebody posted on FB ;) ) by Dr. William H. Gates III; one of the founders of Microsoft Corp. more popularly known as Bill Gates. He says, "increasing the odds, doesn't guarantee success". So, what does?
If you think of it, the "odds" of something happening is not always a very understandable statement. I mean, even when the objective sciences are concerned, lets say weather prediction, they say the chances of raining during a certain period of is some x % or say 30%. What does that statement mean in itself ?
I have had discussion with people as to what does that mean, really mean? The boundary conditions are quiet understandable. I know, it is not gonna rain in that period, say today, if that number were 0% or that it will rain for sure if it were 100%. But anything in between is such a abstract thought in some ways.
The so called frequentist approach perhaps would say that, given the measurements of certain parameters (I don't know what they should be, but I can guess, wind speed, direction, humidity, cloud cover, etc are some of those parameters), on an average, 3 out of the 10 days such conditions exist, "it will rain". Or does it mean, "it has rained" ! ( a historic statement). Is that universal phenomenon or local the place! How old and accurate is this number? Why should I even trust it? How often does it go wrong? (Frequentist question again!)
And as some other people refer to, a subjective definition as to the "odds" or the "chance" of it raining today is 30%. I always have a hard time wrapping my head around such statements! For one, we would not always know the alternate options. I mean, How many other conditions are there besides raining! If there is a 30% chance of raining, what are the chances it will be cloudy? How cloudy is cloudy?
But do common people, much less a guy who is doing statistical signal processing for a living, understand this? If they don't what's the point of giving that information. In the town I live, the weather prediction is so very inaccurate, people know much better looking outside that on the computer or TV. I don't think they really don't have a good weather model for mountain valley (much like a bowl) geography of Laramie (plus the altitude and latitude).
So, coming back to MLE, what does MLE guarantee, if at all anything? I am not sure if it guarantees anything at all. After-all, by definition, it is an estimate. Perhaps, one can only hope that one is on the correct side of things and by chance is at the right place at the right time. Perhaps, it boils down to,
कर्मण्येवाधिकारस्ते मा फलेषु कदाचना
मा कर्मफलह्तुर्भुर्मा ते संगोस्त्वकर्मनी ||
It is only appropriate to just keep doing what you do the best, you are maximizing the likelihood of the results by doing that, perhaps, you will reach a point where you converge under probability! (I had to end it in geeky phrase :P )
An awesome book, explains statistics in simple, humorous words in terms of small stories, analogies and examples. It is really nice to discover how badly people represent (especially when marketing and money is involved) to make it look better that it actually is. One example is, the state tax in a State X is only .1% more than state tax in a State Y. But in fact state tax in the State X is 1.1% (of total value) and State tax in State Y is 1% (of total value) which is actually 10% more, in total tax paid!!
also see: http://en.wikipedia.org/wiki/Convergence_of_random_variables#Almost_sure_convergence for convergence in probability.
My blog on MS bike ride is still pending, sorry about that, I am still waiting on some pics.
I did my first 2 PhD preliminary exams last week, it has been a very good experience with the exam. It really feels good to sit and think on problems that are not necessarily new and unsolved but not in the popular literature or commonly encountered (and some intentionally designed to be confusing!!). It is nice (after it is done) to have to choose a certain word over the other, a certain phrase over the other and a certain logic over the other and prepare to defend the whole thing!