tag:blogger.com,1999:blog-842965756326639856.post3451892779306584853..comments2017-08-17T07:30:38.558-07:00Comments on Eric Jang: A Beginner's Guide to Variational Methods: Mean-Field ApproximationEricnoreply@blogger.comBlogger17125tag:blogger.com,1999:blog-842965756326639856.post-890358475673304452017-06-04T03:42:26.678-07:002017-06-04T03:42:26.678-07:00Do you mind explaining where that negative comes f...Do you mind explaining where that negative comes from? I was anticipating a plus... angusturner27https://www.blogger.com/profile/07981384823717923666noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-53383152898309186372017-05-12T13:01:16.053-07:002017-05-12T13:01:16.053-07:00I believe the last formula for reverse KL should b...I believe the last formula for reverse KL should be an expectation over q, not over p. Great post. Thanks for your effort.mathnathanhttps://www.blogger.com/profile/02594451096636042529noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-7683365764424279032017-05-07T02:53:22.770-07:002017-05-07T02:53:22.770-07:00Thanks for this, it is a key resource for our read...Thanks for this, it is a key resource for our reading group discussion on VAE today https://github.com/p-i-/machinelearning-IRC-freenode/blob/master/ReadingGroup/README.mdSunFish7https://www.blogger.com/profile/16021806731509723216noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-43732015586799387982017-05-07T02:52:09.541-07:002017-05-07T02:52:09.541-07:00Probabilities sum to 1. i.e. Given a probability d...Probabilities sum to 1. i.e. Given a probability distribution q over Z, summing q(z) over all possible z in Z must give 1.SunFish7https://www.blogger.com/profile/16021806731509723216noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-10751644964120852972017-05-03T16:04:34.950-07:002017-05-03T16:04:34.950-07:00Hi, can you explain me the relation of the sum ove...Hi, can you explain me the relation of the sum over q(z) equal to 1 in equation (1)?. Thanks, I don't catch it. Magdiel JimĂ©nez Guarneroshttps://www.blogger.com/profile/10010153146652007747noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-72041606508229268782017-04-25T22:09:19.233-07:002017-04-25T22:09:19.233-07:00I read a few blogs/articles/slides about variation...I read a few blogs/articles/slides about variational autoencoders, and I personally think this is the best one. The key ideas are pointed out clearly. The technical terms(e.g., ELBO) are well explained, too. Thanks so much. sutonyhttps://www.blogger.com/profile/04437574984914675718noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-62035267000117572992017-02-24T14:42:55.131-08:002017-02-24T14:42:55.131-08:00On the off chance that you have not exploited surv...On the off chance that you have not exploited surveillance cameras to ensure your property, please consider to begin utilizing them. <a href="http://allbestchoices.com/best-surveillance-system/" rel="nofollow">best home surveillance system</a><br />Aafiya Designerhttps://www.blogger.com/profile/16894361945088187106noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-48025410435810314842016-11-06T22:47:49.612-08:002016-11-06T22:47:49.612-08:00I didn't know that! Thank you for sharing this...I didn't know that! Thank you for sharing this. I hope that interested readers will scroll down and find your comment. Erichttps://www.blogger.com/profile/05932982386234738790noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-941729952203784422016-10-18T13:01:55.459-07:002016-10-18T13:01:55.459-07:00Given the title of your post, it's worth givin...Given the title of your post, it's worth giving some motivation behind the name "mean-field approximation". <br /><br />From a statistical physics point of view, "mean-field" refers to the relaxation of a difficult optimization problem to a simpler one which ignores second-order effects. For example, in the context of graphical models, one can approximate the partition function of a Markov random field via maximization of the Gibbs free energy (i.e., log partition function minus relative entropy) over the set of product measures, which is significantly more tractable than global optimization over the space of all probability measures (see, e.g., M. Mezard and A. Montanari, Sect 4.4.2).<br /><br />From an algorithmic point of view, "mean-field" refers to the naive mean field algorithm for computing marginals of a Markov random field. Recall that the fixed points of the naive mean field algorithm are optimizers of the mean-field approximation to the Gibbs variational problem. This approach is "mean" in that it is the average/expectation/LLN version of the Gibbs sampler, hence ignoring second-order (stochastic) effects (see, e.g., M. Wainwright and M. Jordan, (2.14) and (2.15)).skimhttps://www.blogger.com/profile/17916084961528723939noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-75273101500625964862016-08-15T00:07:42.623-07:002016-08-15T00:07:42.623-07:00This comment has been removed by a blog administrator.David N. Olsonhttps://www.blogger.com/profile/10629316619066644960noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-78644911786482862672016-08-14T19:12:50.703-07:002016-08-14T19:12:50.703-07:00That's correct! Thank you :)That's correct! Thank you :)Erichttps://www.blogger.com/profile/05932982386234738790noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-12313765257209257952016-08-12T11:06:19.340-07:002016-08-12T11:06:19.340-07:00This tutorial is fantastic!
I believe the phrase...This tutorial is fantastic! <br /><br />I believe the phrase "must be strictly greater than" should omit "strictly" seeing as equality could hold according to your definition. Emery Goossenshttps://www.blogger.com/profile/17588008146723254889noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-27658208018056670012016-08-11T05:04:28.048-07:002016-08-11T05:04:28.048-07:00This comment has been removed by a blog administrator.Fahim Leehttps://www.blogger.com/profile/11607453638221692970noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-19195151618321524762016-08-10T05:42:55.105-07:002016-08-10T05:42:55.105-07:00The post is worth reading.The post is worth reading.John Barnesshttps://www.blogger.com/profile/05985210033843770891noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-4599964367319640032016-08-08T23:58:29.778-07:002016-08-08T23:58:29.778-07:00Thanks for the great post, Eric! Do you plan (or h...Thanks for the great post, Eric! Do you plan (or have a link to) to write a simple tutorial to illustrate the VB in practice?Vladislavs Dovgalecshttps://www.blogger.com/profile/03372803047302329128noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-25533068414087510862016-08-08T14:15:46.743-07:002016-08-08T14:15:46.743-07:00Thanks for your sharp eyes! I added the minus in f...Thanks for your sharp eyes! I added the minus in front of the KL term.Erichttps://www.blogger.com/profile/05932982386234738790noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-73969413568521932752016-08-08T11:34:33.759-07:002016-08-08T11:34:33.759-07:00There should be a minus in equation (3) for E[log ...There should be a minus in equation (3) for E[log p(x|z)] i.e. E[ -log p(x|z)] otherwise your definition of KL-divergence isn't consistent. <br /><br />Ankur.Incognitohttps://www.blogger.com/profile/02971376934493359965noreply@blogger.com