13 December 2005

Interesting topics in NLP

Papers for HLT/NAACL are due Friday, but since I'm probably not submitting anything, I'll take this opportunity to write down some thoughts about what NLP seems to care about and what I think it might care about more. Topic-wise, we have several big areas that typically overwhelm conference proceedings:

  • Machine Translation
  • Summarization
  • Language Modeling
  • Information Extraction
  • Parsing
There's obviously a lot more to NLP than these topics, but these by far recieve the highest publication rate at conferences like ACL, NAACL, HLT and COLING. I think it's fairly obvious why. Each has some strong combination of the "required" two aspects for publication at these conferences:
  • Importance (or strong community beliefs thereof)
  • Evaluatability (especially on common data sets with common metrics)
There is a third aspect under which contributions can be measured that doesn't seem to play a strong role in our field right now: Interestingness. The problem is that evaluating interestingness is difficult. As pointed out by Sam Roweis at a discussion at the BayesNLP workshop, this is something that SIGGRAPH is very successful at. Perhaps SIGGRAPH papers aren't on the most important or evaluatable topics, but they are certainly interesting. Putting too much weight on interestingness is probably a bad idea, especially if it completely forgoes the other two aspects. But maybe we'd be better off as a community if we had one or two tracks (or perhaps a workshop) that focused exclusively on interesting, new problems, even if they fumble a bit on evaluation and obvious short-term importance. Some topics I care about (i.e., think are interesting) that are not commonly seen at ACL-like conferences are:
  • User modeling or personalization in general
  • CogSci-oriented human-learning or -usage of language
  • Situated language experiments, in real or simulated worlds
  • Super-sentence models of coherence, cohesion, relation between documents
  • Tasks involving multiple media (text + {video, images, charts/tables, ...})
I don't expect these thing to actually start appearing soon, and I know there are conferences that focus more on each aspect individually, but I think it's time that our community as a whole starts paying more serious attention to what goes on in these areas.

26 comments:

  1. It seems to me that the NLP field is overrun by computer scientists who like to build text processing systems. To them, it is more compelling to build things that work than to search for cognitive models that explain natural phenomena. Not that there is anything wrong with it.

    ReplyDelete
  2. I agree with the point about "interestingness." Unfortunately, there seems to be an inherent tradeoff between being interesting and having high scores in the evaluation. New, novel ideas often run the risk of underperforming existing systems, which may be highly-tuned. This is especially the case for large systems. For instance, in speech recognition, a researcher may develop a new language model with good perplexity, but after chaining it with the highly integrated acoustic and pronunciation model, give no WER improvements.

    ReplyDelete
  3. I think as a community we're very disenchanted with cool stuff right now, and essentially use a research "search" technique with a constant zero heuristic function: we generally will only accept things that have been shown to work, not things with "promise." I think, and I believe many people agree, that this is leading us to a local maximum. The only way out is for some very strong willed people to forge ahead and ignore the community for a while, essentially applying their own, non-zero heuristic. This seems dumb: I'd like to be able to benefit from what they're doing.

    OTOH, maybe we're not in a local max. Maybe we just haven't found the right local search operations to move higher (and toward more interesting things). Possible, but I tend to doubt it.

    ReplyDelete
  4. 酒店經紀PRETTY GIRL 台北酒店經紀人 ,禮服店 酒店兼差PRETTY GIRL酒店公關 酒店小姐 彩色爆米花酒店兼職,酒店工作 彩色爆米花酒店經紀, 酒店上班,酒店工作 PRETTY GIRL酒店喝酒酒店上班 彩色爆米花台北酒店酒店小姐 PRETTY GIRL酒店上班酒店打工PRETTY GIRL酒店打工酒店經紀 彩色爆米花

    ReplyDelete
  5. I really liked your blog! You have some great content. Please come visit my site contract administration when you got time.

    ReplyDelete
  6. I really liked your post on home financing! if you have more information elsewhere let me know. Please come visit my site international marketing when you got time.

    ReplyDelete
  7. bingo, ireland, irish bingo, bingo in ireland, online bingo, online bingo Ireland, free bingo, bingo in euros, play bingo in euros, bonus bingo, 90 ball bingo, no deposit bingo, free bingo in ireland, free irish bingo, paddypower, rehabbingo, rehab bingo, bingo online, bingo games, internet bingo, irish internet bingo, slot games, online slot games, scratch cards, online scratch cards, free euro bingo, bingo halls in Ireland, play bingo

    ReplyDelete
  8. Couldn't be written any better. Reading this post reminds me of my old room mate! He always kept talking about this. I will forward this article to him. Pretty sure he will have a good read. Thanks for sharing! I would love some feedback on my site landscape designer when you got time.

    ReplyDelete
  9. Excellent article , i just share it with my friend of Italy. I Stumble UP your blog post , you will notice an increase of traffic within 24 hours for targeted people. Cheers . Please come visit my site garden centers give me any valuable feedbacks.

    ReplyDelete
  10. I am deeply in love with every single piece of information you post here. Will be back often to read more updates! Please come visit my site cheap Wholesale Canada Watches when you got time.

    ReplyDelete
  11. Couldn't be written any better. Reading this post reminds me of my old room mate! He always kept talking about this. I will forward this article to him. Pretty sure he will have a good read. Thanks for sharing! I would love some feedback on my site Sports injuries when you got time.

    ReplyDelete
  12. Excellent article , i just share it with my friend of Italy. I Stumble UP your blog post , you will notice an increase of traffic within 24 hours for targeted people. Cheers . Please come visit my site Apartment living\ give me any valuable feedbacks.

    ReplyDelete
  13. I am more interested in Summarization and information extracting.

    dentists edinburgh

    ReplyDelete
  14. I am getting interested in nlp. Can you tell some nice ebooks available on internet so that I could get some more knowledge about it.

    ReplyDelete
  15. This is such a great resource that you are providing and you give it away for free. I love seeing websites that understand the value of providing a quality resource for free. It’s the old what goes around comes around routine.
    child
    abuse defense attorney
    , Patent
    Trademark attorney
    , attorney
    for personal injury
    , attorney
    for child abuse
    , Patent
    Trademark lawyer
    , personal
    injury law attorneys
    , criminal
    law child abuse attorneys
    , Patent
    lawyers Trademark Attorneys
    ,

    ReplyDelete
  16. There are certainly a lot of details like that to take into consideration. That’s a great point to bring up. I offer the thoughts above as general inspiration but clearly there are questions like the one you bring up where the most important thing will be working in honest good faith.
    Baltimore Divorce Attorneys, San
    Francisco Divorce Attorneys
    , Los
    Angeles Divorce Attorneys
    , Dallas
    Divorce Attorneys
    , Chicago
    Divorce Attorneys
    , Austin
    Divorce Attorneys
    , Houston
    Divorce Attorneys
    , wholsale
    clothing

    ReplyDelete
  17. I would just like to say to all my fellow Entrepreneurs…just remember that persistence is the key to success. Personal Injury Attorney Houston

    ReplyDelete
  18. Hi! Thanks for the great information you havr provided! You have touched on crucuial points! I would just like to say to all my fellow Entrepreneurs…just remember that persistence is the key to success. As Winston Churchill once said: “Never, never, never give up!”
    Alabama insurance consultants, Alaska insurance consultants, Arizona insurance consultants, Arkansas insurance consultants, Bail Bonds Referral Services insurance consultants, California insurance consultants, Colorado insurance consultants, Connecticut insurance consultants, Delaware insurance consultants, District Of Colombia insurance consultants, Florida insurance consultants, Georgia insurance consultants,

    ReplyDelete
  19. This is such a great resource that you are providing and you give it away for free. I love seeing websites that understand the value of providing a quality resource for free. It’s the old what goes around comes around routine.
    historic buildings, Canoe
    Dealers, Kayak Rentals, Canoe Trips
    , Heating
    And Air Conditioning
    , Best
    Driving School
    , historic preservation society

    ReplyDelete
  20. I just couldnt leave your website before saying that I really enjoyed the quality information you offer to your visitors... Will be back often to check up on new stuff you post here! Canoe
    And Kayaking
    , Air Conditioning
    Companies
    , Truck Driver Training, historical society museum, Kayak
    Canoe Rentals
    , Air Conditioning
    Maintenance
    ,  Truck Driving Education & Careers

    ReplyDelete
  21. Hi! Thanks for the great information you havr provided! You have touched on crucuial points! I would just like to say to all my fellow Entrepreneurs…just remember that persistence is the key to success. As Winston Churchill once said: “Never, never, never give up!”
    wholesale
    silver chains
    , earring
    wholesale
    , home security alarm system, earring
    jewelry
    , outdoor security systerm China manufacturer, wholesale
    fashion jewelry
    , wholesale
    fashion earring
    , Quebec
    QC Phone Directory

    ReplyDelete