Comments on Quick History 2: GLMs, R and large data setsTypePad2014-05-21T20:39:57ZBlog Administratorhttps://blog.revolutionanalytics.com/tag:typepad.com,2003:https://blog.revolutionanalytics.com/2014/05/quick-history-2-glms-r-and-large-data-sets/comments/atom.xml/Jeff Stuckman commented on 'Quick History 2: GLMs, R and large data sets'tag:typepad.com,2003:6a010534b1db25970b01a3fd101afe970b2014-05-25T16:18:50Z2014-05-27T17:51:43ZJeff StuckmanAside from the inability of glm() to cope with very large datasets, another problem I've had with training GLMs is...<p>Aside from the inability of glm() to cope with very large datasets, another problem I've had with training GLMs is dealing with cases where the underlying optimization algorithm fails, possibly due to an inappropriate choice of initial parameters.</p>
<p>I've had some success with a technique that I saw on a blog somewhere -- selecting a small subset of the data, training an initial model on the subset, and using that model's fitted parameters as the initial parameters to the full GLM. However, I've also noticed that bigglm has much more success in fitting these "difficult" models, possibly due to its underlying algorithm being different.</p>
<p>So, that's another reason to choose bigglm over glm.</p>