Comments on Large scale optimization with optim() and RevoScaleRTypePad2013-12-31T22:46:42ZBlog Administratorhttp://blog.revolutionanalytics.com/tag:typepad.com,2003:http://blog.revolutionanalytics.com/2014/01/large-scale-optimization-with-optim-and-revoscaler/comments/atom.xml/Derek Norton commented on 'Large scale optimization with optim() and RevoScaleR'tag:typepad.com,2003:6a010534b1db25970b019b04da71fc970d2014-01-16T16:21:36Z2014-01-16T16:21:39ZDerek Nortonhttp://profile.typepad.com/dereknortonRichard, thanks for the comment, however, your point about sufficient statistics is already mentioned in the post, i.e. "This is...<p>Richard, thanks for the comment, however, your point about sufficient statistics is already mentioned in the post, i.e. "This is of course silly since there is a closed form for the mean and standard deviation, but it does help clearly demonstrate optimization scaled up."</p>Richard commented on 'Large scale optimization with optim() and RevoScaleR'tag:typepad.com,2003:6a010534b1db25970b01a3fc4de699970b2014-01-16T16:04:05Z2014-01-24T00:51:03ZRichardthis is a nice parallel method. however with simple models like a normal distribution, you should check if sufficient statistics...<p>this is a nice parallel method. however with simple models like a normal distribution, you should check if sufficient statistics exist. for the normal distribution, there is no need to loop through each data pointt, get each data points log pdf, and sum the resuts. if you work out the math you will find that you only need the sum and sum of squares from the entire data set, regardless of size, to get the likelihood. therefore you only need one initial loop to get the sufficient statistics, as opposed to looping for each optimization iteration.</p>
<p>however this is dependent on the underlying statistical model. cimplicated or convoluted likelihoods or distributions without a full set of sufficient statistics will need something like this.</p>omar commented on 'Large scale optimization with optim() and RevoScaleR'tag:typepad.com,2003:6a010534b1db25970b019b04b8c4a5970d2014-01-14T04:24:44Z2014-01-24T00:51:03Zomarinteresting<p>interesting</p>Randcore commented on 'Large scale optimization with optim() and RevoScaleR'tag:typepad.com,2003:6a010534b1db25970b01a510a13b25970c2014-01-09T17:06:27Z2014-01-24T00:51:03ZRandcoreThis is very cool! Could you do an example of GMM following the same principles?<p>This is very cool! Could you do an example of GMM following the same principles?</p>