tag:blogger.com,1999:blog-842965756326639856.post3451892779306584853..comments2017-04-25T22:24:44.582-07:00Comments on Eric Jang: A Beginner's Guide to Variational Methods: Mean-Field ApproximationEricnoreply@blogger.comBlogger12125tag:blogger.com,1999:blog-842965756326639856.post-72041606508229268782017-04-25T22:09:19.233-07:002017-04-25T22:09:19.233-07:00I read a few blogs/articles/slides about variation...I read a few blogs/articles/slides about variational autoencoders, and I personally think this is the best one. The key ideas are pointed out clearly. The technical terms(e.g., ELBO) are well explained, too. Thanks so much. sutonyhttp://www.blogger.com/profile/04437574984914675718noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-62035267000117572992017-02-24T14:42:55.131-08:002017-02-24T14:42:55.131-08:00On the off chance that you have not exploited surv...On the off chance that you have not exploited surveillance cameras to ensure your property, please consider to begin utilizing them. <a href="http://allbestchoices.com/best-surveillance-system/" rel="nofollow">best home surveillance system</a><br />Aafiya Designerhttp://www.blogger.com/profile/16894361945088187106noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-48025410435810314842016-11-06T22:47:49.612-08:002016-11-06T22:47:49.612-08:00I didn't know that! Thank you for sharing this...I didn't know that! Thank you for sharing this. I hope that interested readers will scroll down and find your comment. Erichttp://www.blogger.com/profile/05932982386234738790noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-941729952203784422016-10-18T13:01:55.459-07:002016-10-18T13:01:55.459-07:00Given the title of your post, it's worth givin...Given the title of your post, it's worth giving some motivation behind the name "mean-field approximation". <br /><br />From a statistical physics point of view, "mean-field" refers to the relaxation of a difficult optimization problem to a simpler one which ignores second-order effects. For example, in the context of graphical models, one can approximate the partition function of a Markov random field via maximization of the Gibbs free energy (i.e., log partition function minus relative entropy) over the set of product measures, which is significantly more tractable than global optimization over the space of all probability measures (see, e.g., M. Mezard and A. Montanari, Sect 4.4.2).<br /><br />From an algorithmic point of view, "mean-field" refers to the naive mean field algorithm for computing marginals of a Markov random field. Recall that the fixed points of the naive mean field algorithm are optimizers of the mean-field approximation to the Gibbs variational problem. This approach is "mean" in that it is the average/expectation/LLN version of the Gibbs sampler, hence ignoring second-order (stochastic) effects (see, e.g., M. Wainwright and M. Jordan, (2.14) and (2.15)).skimhttp://www.blogger.com/profile/17916084961528723939noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-75273101500625964862016-08-15T00:07:42.623-07:002016-08-15T00:07:42.623-07:00This comment has been removed by a blog administrator.David N. Olsonhttp://www.blogger.com/profile/10629316619066644960noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-78644911786482862672016-08-14T19:12:50.703-07:002016-08-14T19:12:50.703-07:00That's correct! Thank you :)That's correct! Thank you :)Erichttp://www.blogger.com/profile/05932982386234738790noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-12313765257209257952016-08-12T11:06:19.340-07:002016-08-12T11:06:19.340-07:00This tutorial is fantastic!
I believe the phrase...This tutorial is fantastic! <br /><br />I believe the phrase "must be strictly greater than" should omit "strictly" seeing as equality could hold according to your definition. Emery Goossenshttp://www.blogger.com/profile/17588008146723254889noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-27658208018056670012016-08-11T05:04:28.048-07:002016-08-11T05:04:28.048-07:00This comment has been removed by a blog administrator.Fahim Leehttp://www.blogger.com/profile/11607453638221692970noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-19195151618321524762016-08-10T05:42:55.105-07:002016-08-10T05:42:55.105-07:00The post is worth reading.The post is worth reading.John Barnesshttp://www.blogger.com/profile/05985210033843770891noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-4599964367319640032016-08-08T23:58:29.778-07:002016-08-08T23:58:29.778-07:00Thanks for the great post, Eric! Do you plan (or h...Thanks for the great post, Eric! Do you plan (or have a link to) to write a simple tutorial to illustrate the VB in practice?Vladislavs Dovgalecshttp://www.blogger.com/profile/03372803047302329128noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-25533068414087510862016-08-08T14:15:46.743-07:002016-08-08T14:15:46.743-07:00Thanks for your sharp eyes! I added the minus in f...Thanks for your sharp eyes! I added the minus in front of the KL term.Erichttp://www.blogger.com/profile/05932982386234738790noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-73969413568521932752016-08-08T11:34:33.759-07:002016-08-08T11:34:33.759-07:00There should be a minus in equation (3) for E[log ...There should be a minus in equation (3) for E[log p(x|z)] i.e. E[ -log p(x|z)] otherwise your definition of KL-divergence isn't consistent. <br /><br />Ankur.Incognitohttp://www.blogger.com/profile/02971376934493359965noreply@blogger.com