13 December 2005

Interesting topics in NLP

Papers for HLT/NAACL are due Friday, but since I'm probably not submitting anything, I'll take this opportunity to write down some thoughts about what NLP seems to care about and what I think it might care about more. Topic-wise, we have several big areas that typically overwhelm conference proceedings:

  • Machine Translation
  • Summarization
  • Language Modeling
  • Information Extraction
  • Parsing
There's obviously a lot more to NLP than these topics, but these by far recieve the highest publication rate at conferences like ACL, NAACL, HLT and COLING. I think it's fairly obvious why. Each has some strong combination of the "required" two aspects for publication at these conferences:
  • Importance (or strong community beliefs thereof)
  • Evaluatability (especially on common data sets with common metrics)
There is a third aspect under which contributions can be measured that doesn't seem to play a strong role in our field right now: Interestingness. The problem is that evaluating interestingness is difficult. As pointed out by Sam Roweis at a discussion at the BayesNLP workshop, this is something that SIGGRAPH is very successful at. Perhaps SIGGRAPH papers aren't on the most important or evaluatable topics, but they are certainly interesting. Putting too much weight on interestingness is probably a bad idea, especially if it completely forgoes the other two aspects. But maybe we'd be better off as a community if we had one or two tracks (or perhaps a workshop) that focused exclusively on interesting, new problems, even if they fumble a bit on evaluation and obvious short-term importance. Some topics I care about (i.e., think are interesting) that are not commonly seen at ACL-like conferences are:
  • User modeling or personalization in general
  • CogSci-oriented human-learning or -usage of language
  • Situated language experiments, in real or simulated worlds
  • Super-sentence models of coherence, cohesion, relation between documents
  • Tasks involving multiple media (text + {video, images, charts/tables, ...})
I don't expect these thing to actually start appearing soon, and I know there are conferences that focus more on each aspect individually, but I think it's time that our community as a whole starts paying more serious attention to what goes on in these areas.

26 comments:

Anonymous said...

It seems to me that the NLP field is overrun by computer scientists who like to build text processing systems. To them, it is more compelling to build things that work than to search for cognitive models that explain natural phenomena. Not that there is anything wrong with it.

Kevin Duh said...

I agree with the point about "interestingness." Unfortunately, there seems to be an inherent tradeoff between being interesting and having high scores in the evaluation. New, novel ideas often run the risk of underperforming existing systems, which may be highly-tuned. This is especially the case for large systems. For instance, in speech recognition, a researcher may develop a new language model with good perplexity, but after chaining it with the highly integrated acoustic and pronunciation model, give no WER improvements.

hal said...

I think as a community we're very disenchanted with cool stuff right now, and essentially use a research "search" technique with a constant zero heuristic function: we generally will only accept things that have been shown to work, not things with "promise." I think, and I believe many people agree, that this is leading us to a local maximum. The only way out is for some very strong willed people to forge ahead and ignore the community for a while, essentially applying their own, non-zero heuristic. This seems dumb: I'd like to be able to benefit from what they're doing.

OTOH, maybe we're not in a local max. Maybe we just haven't found the right local search operations to move higher (and toward more interesting things). Possible, but I tend to doubt it.

Anonymous said...

酒店經紀PRETTY GIRL 台北酒店經紀人 ,禮服店 酒店兼差PRETTY GIRL酒店公關 酒店小姐 彩色爆米花酒店兼職,酒店工作 彩色爆米花酒店經紀, 酒店上班,酒店工作 PRETTY GIRL酒店喝酒酒店上班 彩色爆米花台北酒店酒店小姐 PRETTY GIRL酒店上班酒店打工PRETTY GIRL酒店打工酒店經紀 彩色爆米花

enterprise contract management said...

I really liked your blog! You have some great content. Please come visit my site contract administration when you got time.

industrial goods marketing said...

I really liked your post on home financing! if you have more information elsewhere let me know. Please come visit my site international marketing when you got time.

johnsmith said...

bingo, ireland, irish bingo, bingo in ireland, online bingo, online bingo Ireland, free bingo, bingo in euros, play bingo in euros, bonus bingo, 90 ball bingo, no deposit bingo, free bingo in ireland, free irish bingo, paddypower, rehabbingo, rehab bingo, bingo online, bingo games, internet bingo, irish internet bingo, slot games, online slot games, scratch cards, online scratch cards, free euro bingo, bingo halls in Ireland, play bingo

landscape architects said...

Couldn't be written any better. Reading this post reminds me of my old room mate! He always kept talking about this. I will forward this article to him. Pretty sure he will have a good read. Thanks for sharing! I would love some feedback on my site landscape designer when you got time.

garden supplies wholesale said...

Excellent article , i just share it with my friend of Italy. I Stumble UP your blog post , you will notice an increase of traffic within 24 hours for targeted people. Cheers . Please come visit my site garden centers give me any valuable feedbacks.

Nunavut said...

I am deeply in love with every single piece of information you post here. Will be back often to read more updates! Please come visit my site cheap Wholesale Canada Watches when you got time.

Sport Injury said...

Couldn't be written any better. Reading this post reminds me of my old room mate! He always kept talking about this. I will forward this article to him. Pretty sure he will have a good read. Thanks for sharing! I would love some feedback on my site Sports injuries when you got time.

Apartments Living said...

Excellent article , i just share it with my friend of Italy. I Stumble UP your blog post , you will notice an increase of traffic within 24 hours for targeted people. Cheers . Please come visit my site Apartment living\ give me any valuable feedbacks.

Ericwipe287 said...

I am more interested in Summarization and information extracting.

dentists edinburgh

Gutter Cleaning said...

I am getting interested in nlp. Can you tell some nice ebooks available on internet so that I could get some more knowledge about it.

Maryland Personal Injury Attorneys said...

This is such a great resource that you are providing and you give it away for free. I love seeing websites that understand the value of providing a quality resource for free. It’s the old what goes around comes around routine.
Maryland Attorney Yellow Pages, Attorneys
Maryland
, Maryland Corporate Business Attorneys, Maryland Corporate Finance & Securities Attorneys, Maryland Creditors&#; Rights Attorneys, Maryland Criminal Law Attorneys, Maryland Custody & Support Law Attorneys, Maryland Debt Consolidation Attorneys, Maryland Disability Law Attorneys, Maryland Discrimination & Civil Rights Attorneys, Maryland Divorce & Mediation Services, Maryland Divorce Attorneys, Maryland Election Law Attorneys, Maryland Eminent Domain & Condemnation Attorneys, Maryland Employment & Labor Law Attorneys, Maryland Entertainment & Sports Law Attorneys, Maryland Environmental & Natural Resources Attorneys, Maryland Estate Planning & Administration Attorneys, Maryland Expert Testimony Services, Maryland Family Law Attorneys, Maryland Firearm & Gun Law Attorneys, Maryland Franchise & Licensing Law Attorneys, Maryland General Practice Attorneys, Maryland Government Contracts & Claims Attorneys, Maryland Guardianship & Conservatorship Attorneys, Maryland Health Care Law Attorneys

Michigan Personal Injury Attorneys said...

There are certainly a lot of details like that to take into consideration. That’s a great point to bring up. I offer the thoughts above as general inspiration but clearly there are questions like the one you bring up where the most important thing will be working in honest good faith.
Michigan Attorney Yellow Pages, Attorneys
Michigan
, Michigan Corporate Business Attorneys, Michigan Corporate Finance & Securities Attorneys, Michigan Creditors&#; Rights Attorneys, Michigan Criminal Law Attorneys, Michigan Custody & Support Law Attorneys, Michigan Debt Consolidation Attorneys, Michigan Disability Law Attorneys, Michigan Discrimination & Civil Rights Attorneys, Michigan Divorce & Mediation Services, Michigan Divorce Attorneys, Michigan Election Law Attorneys, Michigan Eminent Domain & Condemnation Attorneys, Michigan Employment & Labor Law Attorneys, Michigan Entertainment & Sports Law Attorneys, Michigan Environmental & Natural Resources Attorneys, Michigan Estate Planning & Administration Attorneys, Michigan Expert Testimony Services, Michigan Family Law Attorneys, Michigan Firearm & Gun Law Attorneys, Michigan Franchise & Licensing Law Attorneys, Michigan General Practice Attorneys, Michigan Government Contracts & Claims Attorneys, Michigan Guardianship & Conservatorship Attorneys, Michigan Health Care Law Attorneys

State Of Maryland Lawyer said...

This is such a great resource that you are providing and you give it away for free. I love seeing websites that understand the value of providing a quality resource for free. It’s the old what goes around comes around routine.
child
abuse defense attorney
, Patent
Trademark attorney
, attorney
for personal injury
, attorney
for child abuse
, Patent
Trademark lawyer
, personal
injury law attorneys
, criminal
law child abuse attorneys
, Patent
lawyers Trademark Attorneys
,

State Of Michigan Lawyer said...

There are certainly a lot of details like that to take into consideration. That’s a great point to bring up. I offer the thoughts above as general inspiration but clearly there are questions like the one you bring up where the most important thing will be working in honest good faith.
Baltimore Divorce Attorneys, San
Francisco Divorce Attorneys
, Los
Angeles Divorce Attorneys
, Dallas
Divorce Attorneys
, Chicago
Divorce Attorneys
, Austin
Divorce Attorneys
, Houston
Divorce Attorneys
, wholsale
clothing

Arizona foreclosures said...

I would just like to say to all my fellow Entrepreneurs…just remember that persistence is the key to success. Personal Injury Attorney Houston

Unknown said...

found your site on del.icio.us today and really liked it.. i bookmarked it and will be back to check it out some more later ..

South Dakota home insurance, Tennessee home insurance, Texas home insurance, Utah home insurance, Vermont home insurance, Virginia home insurance, Washington home insurance, West Virginia home insurance, Wisconsin home insurance, Wyoming home insurance

Unknown said...

Hi! Thanks for the great information you havr provided! You have touched on crucuial points! I would just like to say to all my fellow Entrepreneurs…just remember that persistence is the key to success. As Winston Churchill once said: “Never, never, never give up!”
Alabama insurance consultants, Alaska insurance consultants, Arizona insurance consultants, Arkansas insurance consultants, Bail Bonds Referral Services insurance consultants, California insurance consultants, Colorado insurance consultants, Connecticut insurance consultants, Delaware insurance consultants, District Of Colombia insurance consultants, Florida insurance consultants, Georgia insurance consultants,

Unknown said...

Michigan insurance agencies and brokers. Michigan car insurance, home owner insurance, health insurance, life insurance and more.
Michigan
Disability Insurance
, Michigan
Earthquake insurance
, Michigan
Estate Planning Insurance And Legal Services
, Michigan
Farm, Crop, & Livestock Insurance
, Michigan
Financial Guaranty & Life Insurance
, Michigan
Fire Insurance
, Michigan
Fleet Insurance
, Michigan
Flood Insurance
, Michigan
Foreign Insurance
, Michigan
Franchising
, Michigan
Government Insurance Carriers
, Michigan
Group Insurance
, Michigan
Health Insurance

Unknown said...

This is such a great resource that you are providing and you give it away for free. I love seeing websites that understand the value of providing a quality resource for free. It’s the old what goes around comes around routine.
historic buildings, Canoe
Dealers, Kayak Rentals, Canoe Trips
, Heating
And Air Conditioning
, Best
Driving School
, historic preservation society

Unknown said...

I just couldnt leave your website before saying that I really enjoyed the quality information you offer to your visitors... Will be back often to check up on new stuff you post here! Canoe
And Kayaking
, Air Conditioning
Companies
, Truck Driver Training, historical society museum, Kayak
Canoe Rentals
, Air Conditioning
Maintenance
,  Truck Driving Education & Careers

Anonymous said...

found your site on del.icio.us today and really liked it.. i bookmarked it and will be back to check it out some more later ..


wholesale
earring
, security cameras systems, silver chain
wholesale
, wholesale earring jewelry, car security systerm China manufacturer, silver necklace
supplier
, wholesale earrings, digital security systerm China manufacturer

Anonymous said...

Hi! Thanks for the great information you havr provided! You have touched on crucuial points! I would just like to say to all my fellow Entrepreneurs…just remember that persistence is the key to success. As Winston Churchill once said: “Never, never, never give up!”
wholesale
silver chains
, earring
wholesale
, home security alarm system, earring
jewelry
, outdoor security systerm China manufacturer, wholesale
fashion jewelry
, wholesale
fashion earring
, Quebec
QC Phone Directory