17 September 2006

Statistical NLP is not NLP but just Statistics?

bact' brings up an interesting point, perhaps more provocative than my original (intended-to-be provocative) pseudo-question. To quote, he says:

and some also said,
statistical natural language processing is not language processing at all, only statistics :P

My impression is that the only sense in which this sentence is true is if you insist that what goes on inside the black box of statistical NLP is somehow explaining what goes on inside our heads.  I see it as essentially parallel to the argument against "neural-style" machine learning.  Some neural networks people used to claim (some still do, I hear) that what happens in an artificial neural net is essentially the same as what goes on in our minds.  My impression (though this is now outside what I really know for sure) is that most cognitive scientists would strongly disagree with this claim.  I get the sense that the majority of people who use NNets in practice use them because they work well, not out of some desire to mimic what goes on in our heads.

I feel the same is probably true for most statistical NLP.  I don't know of anyone who would claim that when people parse sentences they do chart parsing (I know some people claim something more along the lines of incremental parsing actually does happen and this seems somewhat plausible to me).  Or that when people translate sentences they apply IBM Model 4 :).

On the other hand, the alternative to statistical NLP is essentially rule-based NLP.  I have an equally hard time believing that we behave simply as rule processing machines when parsing or translating, and that we efficiently store and search through millions of rules in order to do processing.  In fact, I think I have a harder time believing this than believing the model 4 story :P.

Taking a step back, it seems that there are several goals one can have with dealing with language on a computer.  One can be trying to carry out tasks that have to do with language, which I typically refer to as NLP.  Alternatively, one can be trying to model how humans work with language.  I would probably call this CogNLP or something like that.  One could instead try to use computers and language data to uncover "truths" about language.  This is typically considered computational linguistics.  I don't think any of these goals is a priori better than the others, but they are very different.  My general feeling is that NLPers cannot solve all problems, CogNLPers don't really know what goes on in our minds and CLers are a long way from understanding how language functions.  Given this, I think it's usually best to confine a particular piece of work to one of the fields, since trying to solve two or three at a time is likely going to basically be impossible.

16 comments:

Anonymous said...

On the other hand, the alternative to statistical NLP is essentially rule-based NLP.

This statement seems disingenuous. There really are to be a continuum of approaches that use more and more linguistic information. Take, for example, the famous machine translation pyramid. At the bottom, MT approaches using a bag of words model or low-order HMMs really do seem to be "only statistics", while higher up approaches such as syntax-based MT or interlingual-based MT may involve statistics but try to use actual features of natural language.

hal said...

Hrm, I guess this is up to how you interpret the original statement. "Modern" syntactic MT (ala David Chiang's Hiero, Dekai Wu's ITG, Dan Melamed's groups' work and the work from ISI) I would consider statistical NLP, exactly in the same sense that bag-of-words or phrase-based mdoels are statistical NLP. I would also consider things like example based MT (EMBT) to be a form of statistical NLP (albeit one that doesn't use probabilities). On the other hand, I would consider things like the KANT project from CMU (interlingua MT) to not be statistical NLP.

If by statistical NLP you mean bag-of-words NLP, then yes, I would agree, that's not really language processing. But there's a lot more you can do with statistics than just HMMs.

Ignacio Nicolás Rodríguez said...

I don't think any nonstatistical (or statistical, then complemented by something else) method tries to be an imitation of the human process; I believe it's about trading words and the structures they build as "meaning:" that is, not a thing by itself but a pointer.

It's like statistical processing revolves around the pointers while other approaches can go investigate what's pointed to.

hal said...

Ignacio -- thanks for the comment...I think you're right that I was wrong about the goal. I agree that the typical goal is to "meaning." But I guess I don't see why the statistical/non-statistical debate has anything to do with this. My knowledge (which is admittedly slim) of non-statistical methods is that they too "just" push symbols around. I guess I don't see how the non-statistical symbols are any less pointers than the statistical ones.

Anonymous said...

nice site
http://www.aishwaryaraiworld.org/

Anonymous said...

nice site
http://www.fitness-world.org/

Anonymous said...

nice site
http://www.info-language.com/

Anonymous said...

nice site
http://www.outsourcing-world.org/

Anonymous said...

網頁設計,情趣用品,情趣用品,情趣用品,情趣用品
色情遊戲,寄情築園小遊戲,情色文學,一葉情貼圖片區,情惑用品性易購,情人視訊網,辣妹視訊,情色交友,成人論壇,情色論壇,愛情公寓,情色,舊情人,情色貼圖,色情聊天室,色情小說,做愛,做愛影片,性愛

免費視訊聊天室,aio交友愛情館,愛情公寓,一葉情貼圖片區,情色貼圖,情色文學,色情聊天室,情色小說,情色電影,情色論壇,成人論壇,辣妹視訊,視訊聊天室,情色視訊,免費視訊,免費視訊聊天,視訊交友網,視訊聊天室,視訊美女,視訊交友,視訊交友90739,UT聊天室,聊天室,豆豆聊天室,尋夢園聊天室,聊天室尋夢園,080聊天室,080苗栗人聊天室,女同志聊天室,上班族聊天室,小高聊天室 

AV,AV女優
視訊,影音視訊聊天室,視訊交友
視訊,影音視訊聊天室,視訊聊天室,視訊交友,視訊聊天,視訊美女

Anonymous said...

酒店經紀PRETTY GIRL 台北酒店經紀人 ,禮服店 酒店兼差PRETTY GIRL酒店公關 酒店小姐 彩色爆米花酒店兼職,酒店工作 彩色爆米花酒店經紀, 酒店上班,酒店工作 PRETTY GIRL酒店喝酒酒店上班 彩色爆米花台北酒店酒店小姐 PRETTY GIRL酒店上班酒店打工PRETTY GIRL酒店打工酒店經紀 彩色爆米花

Anonymous said...

艾葳酒店經紀提供專業的酒店經紀,酒店上班,酒店打工、兼職、酒店相關知識等酒店相關產業服務,想加入這行業的水水們請找專業又有保障的艾葳酒店經紀公司!
艾葳酒店經紀是合法的公司、我們是不會跟水水簽任何的合約 ( 請放心 ),我們是不會強押水水辛苦工作的薪水,我們絕對不會對任何人公開水水的資料、工作環境高雅時尚,無業績壓力,無脫秀無喝酒壓力,高層次會員制客源,工作輕鬆。
一般的酒店經紀只會在水水們第一次上班和領薪水時出現而已,對水水們的上班安全一點保障都沒有!艾葳酒店經紀公司的水水們上班時全程媽咪作陪,不需擔心!只提供最優質的酒店上班環境、上班條件給水水們。

Unknown said...

one day i went shopping outside,and in an ed hardy store,I found some kinds of ed hardy i love most they are Your website is really good Thank you for the information ed hardy ed hardy ed hardy clothing ed hardy clothing ed hardy shoes ed hardy shoes don ed hardy don ed hardy ed hardy clothes ed hardy clothes ed hardy bags ed hardy bags ed hardy swimwear ed hardy swimwear ed hardy jeans ed hardy jeans ed hardy mens ed hardy mens Thank you for the information

Unknown said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it to a few friends of mine that I know would enjoy reading..
sesli sohbetsesli chatkamerali sohbetseslisohbetsesli sohbet sitelerisesli chat siteleriseslichatsesli sohpetseslisohbet.comsesli chatsesli sohbetkamerali sohbetsesli chatsesli sohbetkamerali sohbet
seslisohbetsesli sohbetkamerali sohbetsesli chatsesli sohbetkamerali sohbet

Anonymous said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it

to a few friends of mine that I know would enjoy reading..
seslisohbet
seslichat
sesli sohbet
sesli chat
sesli
sesli site
görünlütü sohbet
görüntülü chat
kameralı sohbet
kameralı chat
sesli sohbet siteleri
sesli chat siteleri
görüntülü sohbet siteleri
görüntülü chat siteleri
kameralı sohbet siteleri
canlı sohbet
sesli muhabbet
görüntülü muhabbet
kameralı muhabbet
seslidunya
seslisehir
sesli sex

Unknown said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it

to a few friends of mine that I know would enjoy reading..
seslisohbet
seslichat
sesli sohbet
sesli chat
sesli
sesli site
görünlütü sohbet
görüntülü chat
kameralı sohbet
kameralı chat
sesli sohbet siteleri
sesli chat siteleri
sesli muhabbet siteleri
görüntülü sohbet siteleri
görüntülü chat siteleri
görüntülü muhabbet siteleri
kameralı sohbet siteleri
kameralı chat siteleri
kameralı muhabbet siteleri
canlı sohbet
sesli muhabbet
görüntülü muhabbet
kameralı muhabbet
birsesver
birses
seslidunya
seslisehir
sesli sex

Anonymous said...

Southeast pandora jewelry and main Asian countries have twisted pandora bracelets rubies for centuries, but research as to where, cheap pandora charms and how to find more deposits is spare, and production has figured out how pandora uk and mining companies,” Giuliani says, pandora sale to look at exactly the right time and place.Farther cheap pandora investigation of claret formation, based on tectonic scenery, geochemistry, buy pandora fluid inclusions and isotopic ratios, allowed Giuliani’s lineup to remodel a new prototype pandora bracelets uk for the French Institute of Research for Development (IRD) and the National Scientific cheap pandora bracelets Center of Research, two government-sponsored knowledge and technology research pandora bracelets sale institutes that aim to aid in the sustainable pandora bangles development of developing countries.Before the collision of the Eurasian and Indian plates, cheap pandora bangles lagoons or deltas sat in the regions where marble is giant, he says, “and there is the brains pandora necklace to expect that the new thoughts should help development of the artless capital.”