26 June 2007

ACL Business Meeting Results

This afternoon here in Prague was the ACL business meeting. A few interesting points were brought up. As well all know, ACL will be in Columbus, OH next year. It will actually be joint with HLT, which means that (as I previously expected), there won't be a separate HLT next year. Combining with the fact that when ACL is in north america, there is no NAACL, it looks like there will only be one north american conference next year (unless EMNLP--which is now officially a conference--chooses not to co-locate with ACL/HLT). The paper submission deadline looks to be around 11 Jan -- calls will be out in September. EACL 2008 will be in Greece.

The new information: ACL 2009 will be in Singapore, which was one of my two guesses (the other being Beijing). This should be a really nice location, though I'm saddened since I've already been there.

A few changes have been proposed for ACL 2008 in terms of reviewing. None will necessarily happen, but for what it's worth I've added my opinion here. If you have strong feelings, you should contact the board, or perhaps Kathy McKoewn, who is the conference chair.

  • Conditional accepts and/or author feedback. I'd be in favor of doing one of these, but not both (redundancy). I'd prefer author feedback.

  • Increased poster presence with equal footing in the proceedings, ala NIPS. I would also be in favor of this because already we are at four tracks and too much is going on. Alternatively, we could reduce the number of accepted papers, which I actually don't think would be terrible, but going to posters seems like a safer solution. The strongest argument against this is a personality one: ACLers tend to ignore poster sessions. Something would have to be doing about this. Spotlights may help.

  • Wildcards from senior members. The idea would be that "senior" (however defined?) members would be able to play a single wildcard to accept an otherwise controversial paper. I'm probably not in favor of this, partially because it seems to introduce annoying political issues "What? I'm not senior enough for you?" (I wouldn't say that, since I'm not, but...); partially because it seems that this is essentially already the job of area chairs. There may be a problem here, but it seems that there are better, more direct solutions.

  • Something having to do with extra reviewing of borderline papers. I didn't quite get what was meant here; it didn't seem that the proposal was to have fewer than 3 reviews, but to ask for more in case of confusion. I would actually argue for being more extreme: have a single (maybe 2) reviewer to an initial round of rejects and then get three reviews only for those papers that have any chance at all of being accepted. I doubt this idea will fly, though, but it would be interesting to check in previous papers how many got in that had one reviewer give a really bad score.... how many got in that two reviewers gave a really bad score. If these numbers are really really low, then it should be safe. Anyone have access to this data???
Finally, we talked about the "grassroots" efforts. The proposals were: archive videos, augment the anthology to include link structure, augmenting the anthology with tech reports and journals (given permission from publishers), and ours to make CL open access. Speaking with respect to ours, the only big complains were with respect to typesetting information, but several people did voice support, both in the meeting and in person. I remain hopeful!

I'll post more about technical content after the conference.

5 comments:

Kevin Duh said...

Thanks for the real-time report! This is especially nice for those of us who couldn't make it to Prague. :)

Regarding the point on increasing poster presence--I feel this is an idea worth considering as our conferences continue to grow in size. One idea (borrowed from NIPS) is to require oral presentations to have poster presentations as well, thus making poster presentations the common denominator for all submissions. This encourages everyone to attend the poster session. The oral session is reserved for the few truly outstanding papers of interest to the broad community. In addition, like Hal mentioned, short (1min-5min) spotlight presentations for other good papers can help increase awareness of some posters.

We actually considered doing something like this when we were organizing the ACL 2006 Student Workshop. However, it turned out that "double-booking" a paper for both oral and poster presentation is an non-trivial investment in terms of conference center space and time, and ultimately we had to drop the idea. So I think the double poster+oral presentation idea would work for ACL only if the number of oral papers are reduced. NIPS pulls it off because its oral session is single track and very few papers get this double status.

This leads to the question: do we need 4+ parallel tracks? Personally I would favor fewer tracks, since high number of tracks might inadvertantly cause our field to segregate (i.e. sometimes I feel like ACL is like two conferences: a sub-conference on MT and a sub-conference on everything-else.)

In addition, a method to flatten the perceived difference between posters and oral presentations would be to assign papers based on a random/arbitrary decision, rather than on the reviewer scores. This will no doubt cause some anger initially, but it would erase the current stereotype that oral papers are better than poster papers. I guess the original idea is that oral presentations can reach more audience and thus suits better papers, but I personally believe poster presentation could have just as strong an impact.

In sum, I would argue for reducing the number of parallel tracks, reducing the number or oral presentations, requiring oral presenters to have posters, and making the poster presentations (with spotlight) as the key component of the conference. Anyway, that's my naive opinion.

Min said...

Kevin brought up a point about the poster session. I can bring up
another suggestion that I've seen implemented in JCDL - the idea of a
short oral session to "tease" people to come to the poster session.

At JCDL there's a short (non-tracked) 1-minute madness session that
requires all poster presenters to give a one minute talk to entice
every one to come to the poster session and to give a prelude of what
is to be presented at the poster. The "madness" session incorporates
at most one slide from each presenter (these are all concatenated into
one presentation that is controlled by the session chair). This
bridges into the poster session itself and makes this a more directed
session for attendees, as they've marked off which posters they want
to go to.

Anonymous said...

(aside - Hal, I loved your language modelling talk!) I agree that there is a *lot* going on at this ACL in terms of tracks, and that MT is really dominating - yes, some of it is nice, but it does feel a bit like MT is the centre of everything and the other topics are almost afterthoughts...

Anonymous said...

Thanks Hal, nice reporting, and this is very helpful for those of us who weren't in Prague or weren't at the business meeting.

While you don't say this directly, I assume much of the discussion about posters and tracks and reviewing must be motivated by acceptance rates. ACL and related events are leading the charge down to about a 15% acceptance rate, and I'm not so sure that this is a good thing.

Were acceptance rates discussed? I'd really rather see acceptance rates around 25-30%, simply because when they get much lower than that the reviewing becomes very timid, and you tend to get a program dominated by safe and incremental work, which might be why MT dominated the program at ACL 2007. This isn't to say anything bad about MT, but I don't think the program turned out to be especially representative of the broad range of work going on in the ACL community this time around, and I fear that will continue to happen as long as we accept just 15% of our submissions.

Finally, I think people would attend poster sessions if they are done right. :) Too often poster sessions are organized in such a way that they appear as afterthoughts or fillers, as in the poster session that takes place during a reception or something like that. Make the poster sessions a part of the main schedule, maybe as a track during the day, promoted in the same way that papers are promoted (in the program, complete info, etc.) and I think people will come.

Nice work, and thanks for sharing your experiences from ACL.

Thanks,
Ted

Anonymous said...

酒店經紀PRETTY GIRL 台北酒店經紀人 ,禮服店 酒店兼差PRETTY GIRL酒店公關 酒店小姐 彩色爆米花酒店兼職,酒店工作 彩色爆米花酒店經紀, 酒店上班,酒店工作 PRETTY GIRL酒店喝酒酒店上班 彩色爆米花台北酒店酒店小姐 PRETTY GIRL酒店上班酒店打工PRETTY GIRL酒店打工酒店經紀 彩色爆米花