tag:blogger.com,1999:blog-842965756326639856.post3451892779306584853..comments2023-01-21T03:23:30.658-08:00Comments on Eric Jang: A Beginner's Guide to Variational Methods: Mean-Field ApproximationUnknownnoreply@blogger.comBlogger15125tag:blogger.com,1999:blog-842965756326639856.post-22918246166712074322017-12-16T10:49:37.396-08:002017-12-16T10:49:37.396-08:00I am studying variational Bayes on my own, and thi...I am studying variational Bayes on my own, and this was very helpful. Thank you for writing it Anonymoushttps://www.blogger.com/profile/16913509675688543270noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-890358475673304452017-06-04T03:42:26.678-07:002017-06-04T03:42:26.678-07:00Do you mind explaining where that negative comes f...Do you mind explaining where that negative comes from? I was anticipating a plus... angusturner27https://www.blogger.com/profile/07981384823717923666noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-53383152898309186372017-05-12T13:01:16.053-07:002017-05-12T13:01:16.053-07:00I believe the last formula for reverse KL should b...I believe the last formula for reverse KL should be an expectation over q, not over p. Great post. Thanks for your effort.mathnathanhttps://www.blogger.com/profile/02594451096636042529noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-7683365764424279032017-05-07T02:53:22.770-07:002017-05-07T02:53:22.770-07:00Thanks for this, it is a key resource for our read...Thanks for this, it is a key resource for our reading group discussion on VAE today https://github.com/p-i-/machinelearning-IRC-freenode/blob/master/ReadingGroup/README.mdSunFish7https://www.blogger.com/profile/16021806731509723216noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-43732015586799387982017-05-07T02:52:09.541-07:002017-05-07T02:52:09.541-07:00Probabilities sum to 1. i.e. Given a probability d...Probabilities sum to 1. i.e. Given a probability distribution q over Z, summing q(z) over all possible z in Z must give 1.SunFish7https://www.blogger.com/profile/16021806731509723216noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-10751644964120852972017-05-03T16:04:34.950-07:002017-05-03T16:04:34.950-07:00Hi, can you explain me the relation of the sum ove...Hi, can you explain me the relation of the sum over q(z) equal to 1 in equation (1)?. Thanks, I don't catch it. Anonymoushttps://www.blogger.com/profile/10010153146652007747noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-72041606508229268782017-04-25T22:09:19.233-07:002017-04-25T22:09:19.233-07:00I read a few blogs/articles/slides about variation...I read a few blogs/articles/slides about variational autoencoders, and I personally think this is the best one. The key ideas are pointed out clearly. The technical terms(e.g., ELBO) are well explained, too. Thanks so much. sutonyhttps://www.blogger.com/profile/04437574984914675718noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-62035267000117572992017-02-24T14:42:55.131-08:002017-02-24T14:42:55.131-08:00This comment has been removed by a blog administrator.Anonymoushttps://www.blogger.com/profile/16894361945088187106noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-48025410435810314842016-11-06T22:47:49.612-08:002016-11-06T22:47:49.612-08:00I didn't know that! Thank you for sharing this...I didn't know that! Thank you for sharing this. I hope that interested readers will scroll down and find your comment. Erichttps://www.blogger.com/profile/05932982386234738790noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-941729952203784422016-10-18T13:01:55.459-07:002016-10-18T13:01:55.459-07:00Given the title of your post, it's worth givin...Given the title of your post, it's worth giving some motivation behind the name "mean-field approximation". <br /><br />From a statistical physics point of view, "mean-field" refers to the relaxation of a difficult optimization problem to a simpler one which ignores second-order effects. For example, in the context of graphical models, one can approximate the partition function of a Markov random field via maximization of the Gibbs free energy (i.e., log partition function minus relative entropy) over the set of product measures, which is significantly more tractable than global optimization over the space of all probability measures (see, e.g., M. Mezard and A. Montanari, Sect 4.4.2).<br /><br />From an algorithmic point of view, "mean-field" refers to the naive mean field algorithm for computing marginals of a Markov random field. Recall that the fixed points of the naive mean field algorithm are optimizers of the mean-field approximation to the Gibbs variational problem. This approach is "mean" in that it is the average/expectation/LLN version of the Gibbs sampler, hence ignoring second-order (stochastic) effects (see, e.g., M. Wainwright and M. Jordan, (2.14) and (2.15)).skimhttps://www.blogger.com/profile/17916084961528723939noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-75273101500625964862016-08-15T00:07:42.623-07:002016-08-15T00:07:42.623-07:00This comment has been removed by a blog administrator.Troy Floreshttps://www.blogger.com/profile/10629316619066644960noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-27658208018056670012016-08-11T05:04:28.048-07:002016-08-11T05:04:28.048-07:00This comment has been removed by a blog administrator.Anonymoushttps://www.blogger.com/profile/11607453638221692970noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-4599964367319640032016-08-08T23:58:29.778-07:002016-08-08T23:58:29.778-07:00Thanks for the great post, Eric! Do you plan (or h...Thanks for the great post, Eric! Do you plan (or have a link to) to write a simple tutorial to illustrate the VB in practice?Anonymoushttps://www.blogger.com/profile/03372803047302329128noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-25533068414087510862016-08-08T14:15:46.743-07:002016-08-08T14:15:46.743-07:00Thanks for your sharp eyes! I added the minus in f...Thanks for your sharp eyes! I added the minus in front of the KL term.Erichttps://www.blogger.com/profile/05932982386234738790noreply@blogger.comtag:blogger.com,1999:blog-842965756326639856.post-73969413568521932752016-08-08T11:34:33.759-07:002016-08-08T11:34:33.759-07:00There should be a minus in equation (3) for E[log ...There should be a minus in equation (3) for E[log p(x|z)] i.e. E[ -log p(x|z)] otherwise your definition of KL-divergence isn't consistent. <br /><br />Ankur.Incognitohttps://www.blogger.com/profile/02971376934493359965noreply@blogger.com