tag:blogger.com,1999:blog-19803222.post8858447506976989110..comments2024-03-18T01:45:45.724-06:00Comments on natural language processing blog: Multinomial on a graphhalhttp://www.blogger.com/profile/02162908373916390369noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-19803222.post-21790413882245425362009-05-12T10:53:00.000-06:002009-05-12T10:53:00.000-06:00酒店經紀PRETTY GIRL 台北酒店經紀人 ,禮服店 酒店兼差PRETTY GIRL酒店公關 酒...酒店經紀PRETTY GIRL <A HREF="http://www.taipeilady.com/" REL="nofollow" TITLE="台北酒店經紀人">台北酒店經紀人</A> ,<A HREF="http://tw.myblog.yahoo.com/jw!qZ9n..6QEhhc0LkItOBm/" REL="nofollow" TITLE="禮服店">禮服店</A> 酒店兼差PRETTY GIRL<A HREF="http://www.mashow.org/" REL="nofollow" TITLE="酒店公關">酒店公關</A> 酒店小姐 彩色爆米花<A HREF="http://blog.xuite.net/jkl338801/blog/" REL="nofollow" TITLE="酒店兼職">酒店兼職</A>,酒店工作 彩色爆米花<A HREF="http://tw.myblog.yahoo.com/jw!BIBoU5SeBRs21nb_ajFpncbTqXds" REL="nofollow" TITLE="酒店經紀">酒店經紀</A>, <A HREF="http://mypaper.pchome.com.tw/news/thomsan/3/1310065116/20080905040949/" REL="nofollow" TITLE="酒店上班">酒店上班</A>,酒店工作 PRETTY GIRL<A HREF="http://tw.myblog.yahoo.com/jw!rybqykeeER6TH3AKz1HQ5grm/" REL="nofollow" TITLE="酒店喝酒">酒店喝酒</A>酒店上班 彩色爆米花<A HREF="http://mypaper.pchome.com.tw/news/jkl338801/" REL="nofollow" TITLE="台北酒店">台北酒店</A>酒店小姐 PRETTY GIRL<A HREF="http://www.mashow.org/" REL="nofollow" TITLE="酒店上班">酒店上班</A>酒店打工PRETTY GIRL<A HREF="http://www.tpangel.com/" REL="nofollow" TITLE="酒店打工">酒店打工</A>酒店經紀 彩色爆米花Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-19803222.post-63277868571584767842007-07-13T10:00:00.000-06:002007-07-13T10:00:00.000-06:00Lately, I have been looking at Dirichlet Diffusion...Lately, I have been looking at Dirichlet Diffusion Trees. Although they do not directly match your problem, they may give you some ideas...<BR/><BR/>You find a brief intro and further pointers in Ghahramani's tutorial about Non-parametric Bayesian MethodsAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-19803222.post-80915393047214166752007-06-07T19:15:00.000-06:002007-06-07T19:15:00.000-06:00You may want to try ASCIIMathML. With a little mod...You may want to try ASCIIMathML. With a little modification of your template, you can get better math symbols via LaTex.<BR/><BR/>http://www1.chapman.edu/~jipsen/mathml/asciimath.htmlAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-19803222.post-43114993301121920472007-05-17T08:00:00.000-06:002007-05-17T08:00:00.000-06:00Hi,You might want to take a look at the Multivarai...Hi,<BR/><BR/>You might want to take a look at the Multivaraite Polya distribution, which is used to model "burstiness" in the occurrence of words in documents. See:<BR/><BR/>http://en.wikipedia.org/wiki/Multivariate_Polya_distribution<BR/><BR/>Cheers,<BR/><BR/>-ThomasThomasHhttps://www.blogger.com/profile/09500606983971720730noreply@blogger.comtag:blogger.com,1999:blog-19803222.post-27862816790963364302007-04-17T23:39:00.000-06:002007-04-17T23:39:00.000-06:00I'm never good with Bayesian stuff, so ignore (and...I'm never good with Bayesian stuff, so ignore (and forgive) me if the following questions/suggestions are nonsense: <BR/><BR/>1. Is the difficulty with calculating the posterior with your graph multinomial and dirichlet prior due to the $$N^{-1}(i)$$ in the sum? It seems like if the neighborhood changes depending on how you define the graph, it is impossible to derive a closed form solution for a posterior. No?<BR/><BR/>2. Alternatively, is it possible to use your ontology to constrain the inference in some way, so you can continue using the traditional multinomial+dirichlet Bayesian model? <BR/><BR/>3. Though different, this idea of using graphs to model words/documents remind me of this paper: <A HREF="http://nlp.stanford.edu/kristina/papers/ppwalks.pdf" REL="nofollow"> Toutanova et. al. Learning Random Walks for Inducing Word Dependency Distributions, ICML'04</A>. They have a random walk that computes $$p(w_i|w_{i-1})$$ as the stationary distribution. I'm not sure how to add a prior in this--what does a prior even mean to a random walk? Prior on the initial distribution (which is irrelevant in the limit for most cases)? Or prior on the transition matrix? In your case you wanted a prior on $$\theta$$--but it seems like whatever value it is it'll disappear as we rearch stationary distribution... So I guess what you did makes more sense (defining the probability of a document statically, using a fix neighborhood)....<BR/><BR/>ps. The LaTeX looks fine. It's better than nothing!Kevin Duhhttps://www.blogger.com/profile/07407894290644783502noreply@blogger.comtag:blogger.com,1999:blog-19803222.post-75251947400999525362007-04-17T06:26:00.000-06:002007-04-17T06:26:00.000-06:00I think your graph is a Markov chain. You have a ...I think your graph is a Markov chain. You have a word i, with neighbours j_1, ..., j_N, and probabilities p_1, ..., p_N of transitioning to each j from i. The principle eigenvector of the transition matrix is the stationary distribution of the chain. I.e. the probability, on average, of being in each state (word), which I think it what you are after. Yay for eigenvectors 'cause they made the Google dudes billionaires.<BR/><BR/>Some refs:<BR/><BR/> - The eigenvector / stationary distribution relationship is in any text on Markov chains<BR/><BR/> - More interesting stuff goes under the name of spectral graph theory.<BR/><BR/>You might be interested in first passage times (words between observing word i and word j) and distributions thereof, known as phase-type distributions.<BR/><BR/>You might be interested in spectral clustering, which allows you to find the neighbourhoods on a graph given its transition matrix.<BR/><BR/>A prior on graphs? Probably a dirichlet process of some form.<BR/><BR/>Hope that is on topic and not stuff you already know.Noelhttps://www.blogger.com/profile/09666551093622614632noreply@blogger.comtag:blogger.com,1999:blog-19803222.post-18301207814129338102007-04-12T11:57:00.000-06:002007-04-12T11:57:00.000-06:00YW just suggested that maybe the sum should be ins...YW just suggested that maybe the sum should be inside the parenthesis -- this makes more sense to me, and maybe makes analysis easier.<BR/><BR/>Maybe a lot easier.<BR/><BR/>Definitely a lot easier.<BR/><BR/>Now, I just need refs :P.halhttps://www.blogger.com/profile/02162908373916390369noreply@blogger.comtag:blogger.com,1999:blog-19803222.post-501718073531105552007-04-12T10:24:00.000-06:002007-04-12T10:24:00.000-06:00Blech! That LaTeX is UGLY!Blech! That LaTeX is UGLY!halhttps://www.blogger.com/profile/02162908373916390369noreply@blogger.com