tag:blogger.com,1999:blog-19803222.post3666355107581715427..comments2024-03-18T01:45:45.724-06:00Comments on natural language processing blog: AMR: Not semantics, but close (? maybe ???)halhttp://www.blogger.com/profile/02162908373916390369noreply@blogger.comBlogger9125tag:blogger.com,1999:blog-19803222.post-50682721809000151152015-10-12T21:50:00.457-06:002015-10-12T21:50:00.457-06:00I've been reminded that I promised to post a l...I've been reminded that I promised to post a link to the paper-in-progress I mentioned in my comment from 10/2/14 here. It's been out for a few months now; hopefully still of interest:<br /><br />Bender, Emily M., Dan Flickinger, Stephan Oepen, Woodley Packard and Ann Copestake. 2015. Layers of Interpretation: On Grammar and Compositionality. In Proceedings of the 11th International Conference on Computational Semantics (IWCS 2015), London. pp.239-249. <br />http://aclweb.org/anthology/W/W15/W15-0128.pdf<br />Emily M. Benderhttps://www.blogger.com/profile/12220924508264195256noreply@blogger.comtag:blogger.com,1999:blog-19803222.post-30528165995783928962014-10-06T15:29:02.442-06:002014-10-06T15:29:02.442-06:00Some of the shortcomings Hal described are indeed ...Some of the shortcomings Hal described are indeed simply annotation errors, and other examples are not fully representative of AMR.<br />For example, AMR annotation guidelines specify to include gapped agents in matrix clauses and to capture the semantics in noun-noun compounds if it can be expressed by a (simple) semantic role. And many instances of English "'s" and "of" are expressed using frames and roles other than :mod or :poss.<br /> <br />On the other hand, for practical reasons, AMR indeed is not interlingua. Some concepts and roles remain somewhat shallow, particularly if they would require additional concepts not explicit in the original text. But we do strive to continuously deepen AMRs for common patterns as practically feasible. For example, "the president of the United States, Barack Obama" is no longer annotated as "president :poss country" but as<br /> <br />(p / person :wiki "Barack_Obama" :name ...<br><br /> :ARG0-of (h / have-org-role-91<br><br /> :ARG1 (c / country :wiki "United_States" :name ...)<br><br /> :ARG2 (p2 / president)))<br><br /> <br />Most of the AMRs have only single annotations (which allows us to annotate more AMRs) so they will always be imperfections as for any non-toy-size corpus, but we continuously correct errors that we find, often based on a growing number of automatic checks that we apply to the AMR corpus.<br /> <br />Other examples for non-:poss "'s" and "of":<br /> <br />Snt: Obama's announcement<br><br />(a / announce-01<br><br /> :ARG0-of (p / person :wiki "Barack_Obama" ...))<br><br /> <br />Snt: Indonesia's West Papua province<br><br />(p / province :wiki "West_Papua_(province)" :name ...<br><br /> :location (c / country :wiki "Indonesia" ...))<br><br /> <br />Snt: Wednesday's earthquake<br><br />(e / earthquake<br><br /> :time (d / date-entity :weekday (w / wednesday)))<br><br /> <br />Snt.: the roof of the house<br><br />(r / roof<br><br /> :part-of (h / house))<br><br /> <br />Snt.: two gallons of milk<br><br />(m / milk<br><br /> :quant (v / volume-quantity :quant 2 :unit (g / gallon)))<br><br /> <br />Corrected annotation:<br />Snt.: "... insisted the little prince , who wanted to help him ."<br><br />(i / insist-01<br><br /> :ARG0 (p / prince<br><br /> :mod (l / little)<br><br /> :ARG0-of (w / want-01<br><br /> :ARG0 p<br><br /> :ARG1 (h / help-01<br><br /> :ARG1 (h2 / he))))<br><br /> :ARG1 (...))<br><br /> <br />Corrected annotation:<br />Snt.: glass globe<br><br />(g / globe<br><br /> :consist-of (g2 / glass))<br><br /> <br />For more examples on noun-noun compounds, see http://www.isi.edu/~ulf/amr/lib/amr-dict.html#impliedAnonymoushttps://www.blogger.com/profile/07714968411034185026noreply@blogger.comtag:blogger.com,1999:blog-19803222.post-5360107664836214252014-10-03T11:19:36.799-06:002014-10-03T11:19:36.799-06:00@Emily: can't wait!@Emily: can't wait!halhttps://www.blogger.com/profile/02162908373916390369noreply@blogger.comtag:blogger.com,1999:blog-19803222.post-10825723231556022742014-10-02T13:18:55.406-06:002014-10-02T13:18:55.406-06:00@Hal: I think that the distinction between what&#...@Hal: I think that the distinction between what's sometimes called `standing' meaning (associated with sentence types) and `occasion' or `speaker' meaning (what a speaker is using a sentence to communicate, in a given instance) is critical, and I think that a lot of current work in `semantic processing' in our field doesn't keep this distinction clear. My best understanding of AMR is that they are looking to model sentence meaning (one representation that can be used for multiple different applications), but it seems like a lot of the annotations draw on annotator intuitions about occasion meaning. I'm working on a paper about this and will happily point you to it once it's out!Emily M. Benderhttps://www.blogger.com/profile/12220924508264195256noreply@blogger.comtag:blogger.com,1999:blog-19803222.post-10553069051143414912014-10-01T09:52:53.617-06:002014-10-01T09:52:53.617-06:00@Ray: thanks!
@Emily: haha, thanks for calling me...@Ray: thanks!<br /><br />@Emily: haha, thanks for calling me out on the "closest we have" comment. In retrospect I have no idea why I said that! I know (and love) the ERG. I think comments _are_ the peer-review of blogs and now I can go edit it :).<br /><br />More seriously, what I think I meant (or maybe my rationalization) is that I wanted to stress that semantics != meaning, but since AMR doesn't claim to be semantics, we'll have to settle for "meaning." Does that make sense? I'll go back and clarify.halhttps://www.blogger.com/profile/02162908373916390369noreply@blogger.comtag:blogger.com,1999:blog-19803222.post-4348821044697605902014-09-30T11:09:12.260-06:002014-09-30T11:09:12.260-06:00Hi Hal,
Interesting to see your take on AMR!
M...Hi Hal,<br /><br />Interesting to see your take on AMR! <br /><br />My response is prompted by this comment: "But as semantics is the study of the relationship between signifiers and denotation, [AMR]'s probably the closest we have."<br /><br />I wonder: What other systems of semantic representation did you consider before making that claim? (I know, I know: blog posts aren't peer reviewed publications...)<br /><br />In particular, I'd like to draw your attention to Minimal Recursion Semantics (MRS; Copestake et al 2005) and especially the MRS representations produced by the English Resource Grammar (Flickinger 2000, 2011). <br /><br />The ERG is a broad-coverage, rule-based, linguistically precise grammar for English. You can play with a demo here: <a href="http://erg.delph-in.net/" rel="nofollow">erg.delph-in.net</a><br /><br />We've also started documenting the particular analyses behind the MRS representations:<br /><a href="http://www.lrec-conf.org/proceedings/lrec2014/pdf/562_Paper.pdf" rel="nofollow">http://www.lrec-conf.org/proceedings/lrec2014/pdf/562_Paper.pdf</a><br /><br />If it's not a grammar/parser you want but a treebank (or sembank), check out DeepBank, which has representations produced by the ERG over the same old WSJ text as the PTB: <a href="http://moin.delph-in.net/DeepBank" rel="nofollow">http://moin.delph-in.net/DeepBank</a> This is done with the Redwoods treebanking methodology (Oepen et al 2004), meaning the analyses are all grammar-produced but manually disambiguated. DeepBank was the resource behind the DM representation in <a href="http://alt.qcri.org/semeval2014/task8/" rel="nofollow">2014 SemEval Task 8</a> (on semantic dependency parsing), though that representation is impoverished compared with the full MRS. (The Redwoods treebanks aren't limited to WSJ text---<a href="http://moin.delph-in.net/RedwoodsTop" rel="nofollow">lots of other genres</a> are available too.)<br /><br />I think you'll find that ERG-MRS, as compared to AMR, is much more articulated in the representation of things like control and much more consistent because of the annotation methodology. On the other hand, it doesn't do WSD (beyond that which is morphosyntactically required/constrained), including disambiguating the relationship between the members of noun-noun compounds. (On the futility of that one, see Ó Séaghdha, 2007.)Emily M. Benderhttps://www.blogger.com/profile/12220924508264195256noreply@blogger.comtag:blogger.com,1999:blog-19803222.post-79701092676429961502014-09-30T09:35:10.976-06:002014-09-30T09:35:10.976-06:00For the record, my "$&!#* vector" co...For the record, my "$&!#* vector" comment is from my ACL-14 Workshop on Semantic Parsing talk, the slides are available on the <a href="http://sp14.ws/" rel="nofollow"> workshop website</a>. The full quote was: “You can’t cram the meaning of a whole %&!$# sentence into a single $&!#* vector!” I originally exclaimed this in a moment of jetlagged weakness at a small reading group, frustrated by discussing yet another paper that was trying to do this.Anonymoushttps://www.blogger.com/profile/15048824075235863202noreply@blogger.comtag:blogger.com,1999:blog-19803222.post-90971824335351661322014-09-27T15:30:18.611-06:002014-09-27T15:30:18.611-06:00thanks, Nathan -- that's really interesting. i...thanks, Nathan -- that's really interesting. i especially like the "call to arms" :)<br /><br />it's interesting to think about balancing (a) annotator time, (b) learnability (ie to not get back into the olden days again) and (c) what's actually represented. obviously i don't have an answer :)<br />halhttps://www.blogger.com/profile/02162908373916390369noreply@blogger.comtag:blogger.com,1999:blog-19803222.post-89263146438484279852014-09-27T11:56:12.969-06:002014-09-27T11:56:12.969-06:00Hal, thanks for the insightful post.
First, I'...Hal, thanks for the insightful post.<br /><br />First, I'd like to emphasize that one of the chief goals of AMR (in contrast to the semantic representations of the olden days) is to enable rapid human annotation of corpora with broad coverage. Thus, there are many things about it that are shallower than one might like, such as the lack of any formal relationship between the predicates <b>kill-01</b> and <b>die-01</b>. It is a (sometimes difficult!) tradeoff.<br /><br />Second, a historical note: the <i>Little Prince</i> dataset was originally annotated by various people involved in the design of AMR (including myself). The details of individual annotations should be taken with a planet-sized grain of salt. :) Thankfully, the annotation conventions and tool support have become richer since then, and while some of the improvements have been backported ("retrofitted") to that data, there is a great deal of newer data that should be more consistent. For example, <b>:poss</b> was originally superficially applied to genitive constructions—which, as you point out, is far from ideal—but now the definition of <b>:poss</b> is more semantically restricted (<a href="http://www.isi.edu/~ulf/amr/lib/amr-dict.html#:poss" rel="nofollow">documentation here</a>). In my opinion, <b>:consist-of</b> should have been used for <i>glass globe</i> (it is for some sentences, but not others); and there is now a predicate called <a href="http://www.isi.edu/~ulf/amr/lib/amr-dict.html#have-rel-role-91" rel="nofollow"><b>have-rel-role-91</b></a> for kinship and other person-to-person relations.<br /><br />I guess the broader point is that I hope we can continue to push on AMR where it falls short as a semantic representation, without sacrificing annotator productivity. And in my view, it would also be interesting to explore automatic methods for identifying inconsistencies in richly structured annotations to help with the retrofitting process!Nathan Schneiderhttp://nathan.clnoreply@blogger.com