07 December 2008

Two Reviewer Comments that Stuck with Me

Over the years I've gotten a number of positive reviews and negative reviews. I usually remember few of the details of any more than a few months after the good or bad news. Two reviewer quotes, however, have really stuck with me. They were both in overall positive reviews, though one of them is more negative sounding. Obviously I don't know who wrote them, but they've actually had a strong impact on most of my research and my own reviewing since:

  1. "Other approaches don't have to be bad in order for your approach to be good."
  2. "If I were working in this area, I would want to know about these results."
You can guess what I might have written that had caused this reviewer to make the first comment. And actually I've come to think of these two comments as basically saying the same thing.

I feel like we too often see research in a competitive light. There are two things that can cause this. The first is funding. Short-term funding (in the US) is essentially a zero-sum game, which means that I can win only if you lose. (There are models where this is less true, but usually that has other -- not necessarily desirable -- outcomes.)
Link
The second is the scooping effect: many times when we have (what we think is) a cool idea, we want to "beat" everyone else to the punch. I recall a comment John Langford made on his blog (which I cannot seem to find now because I can't figure out what search terms to use) quite some time ago along these lines... saying that for many problems he doesn't care who finds a solution so long as the solution is found. I usually have mixed feelings. For some topics, I definitely feel this way. Indeed, for some topics I actually don't want to be the person who figures it out, either because I don't feel I have the necessary skills or because I don't have the necessary time. For some topics, I do get a feeling of ownership and really want to be the person who does it. My thesis work was like that, as was my document/abstract alignment work. This is definitely highly personal: I know plenty of people who care a lot more about ownership than I do, and many who care a lot less.

What I took away from this comment is essentially the realization that we are all working toward some vague future goal, which has to do with computationalizing language processing (or some other topic, for the non-NLP audience). Progress is good. If I've done work that has something interesting and novel to say about this goal, then it's not bad -- and is often good -- that this builds on and improves on your work.

32 comments:

DesiLinguist said...

Hal,

That's a very insightful blog post.

In the few years I have been working on MT (I am in the US, so you can pretty much guess the project I am working on), I have encountered both the non-competitive, inclusive, "let's-all-work-together" AND the competitive, "my-foo-works-better-half-a-point-better-than-yours-ha!" sides of research. I realize we can never really do away with the latter---and may be that's good since competition can be a pretty good motivator--but it would be good to see a lot more getting along in our field.

D_K said...

Hi,

Thanks for the post.

Spent about 10 minutes reading your blog post and after-thinking. Came up with a list of article (research?) types:

1) Novel idea applied to an old task
2) Old idea applied to a task, to which it wasn't applied before
3) competitive approach by tuning (experimentally guessing?) some parameter values
4) looking to the past, present and foreseeing future - review

If one positions themselves as a cool originator of - even - a new idea, the life checks it and sends feedback. So need for an extra worry and nail-downing a neighbour.

Tuning (especially, when it is almost ideal and polishing gives a bit more of performance) is like eating grits when the loaf has been eaten by someone else. But life is real (not virtual) and this happens.

Making overview may be considered a waste of time (as no new ideas being brought, hence "less value"), however better structuring gives more clear ways to go.

Adam Lopez said...

These reviewer comments echo Simon Peyton Jones's advice on writing papers (hat tip to Liang Huang):

The truth: credit is not like money

Giving credit to others does not diminish the credit you get from your paper


Now that the season for conference paper writing and reviewing is underway in earnest, this is a good reminder.

ryan said...

+1, great post. insightful and inspiring. thanks!

now, if you'll excuse me, i'm off to do some computationalizing...

Bob Carpenter said...

I think most of what we do is deeply inspired by the scientific zeitgeist.

I often feel that there's a way of modeling data I prefer (in that it feels somehow "right") and I want to write about that perspective, partly to explore it myself and partly to bring others along.

I'm always happy to find someone else has already done something I've been thinking about because it lends credence to my preferences and saves me work. Often they do it a whole lot better than I could have.

When I have a more "creative" idea outside of current thinking, the hard part is finding readers, not preventing competition. And it's not just me! Statistical NLP took a long time to catch on and is still gaining in popularity.

Mberenis said...

I'm doing an experiment with comment posting, and I'd like this blog to participate. Send your blog comments to http://acommentpost.blogspot.com or click below to visit the page. Feel free to dump your comments from your blog there, post as many times as you like, repeat comments, etc. It does not matter. FREE SPEECH ZONE AHEAD! Consider it a dump for comment posts. This will be an interesting experiment.

Post a Comment

Rachel Cotterill said...

I think (as a current PhD student!) that the issue of 'ownership' is a much bigger deal when you're hoping to get a degree for your original contribution... usually I'm the most sharing person, but doing a PhD makes even me a little jealous of my ideas!!!

Anonymous said...

It is the knight noah which makes me very happy these days, my brother says knight gold is his favorite games gold he likes, he usually knight online gold to start his game and most of the time he will win the knight online noah back and give me some cheap knight gold to play the game.

Anonymous said...

網頁設計,情趣用品,情趣用品,情趣用品,情趣用品
色情遊戲,寄情築園小遊戲,情色文學,一葉情貼圖片區,情惑用品性易購,情人視訊網,辣妹視訊,情色交友,成人論壇,情色論壇,愛情公寓,情色,舊情人,情色貼圖,色情聊天室,色情小說,做愛,做愛影片,性愛

免費視訊聊天室,aio交友愛情館,愛情公寓,一葉情貼圖片區,情色貼圖,情色文學,色情聊天室,情色小說,情色電影,情色論壇,成人論壇,辣妹視訊,視訊聊天室,情色視訊,免費視訊,免費視訊聊天,視訊交友網,視訊聊天室,視訊美女,視訊交友,視訊交友90739,UT聊天室,聊天室,豆豆聊天室,尋夢園聊天室,聊天室尋夢園,080聊天室,080苗栗人聊天室,女同志聊天室,上班族聊天室,小高聊天室 

AV,AV女優
視訊,影音視訊聊天室,視訊交友
視訊,影音視訊聊天室,視訊聊天室,視訊交友,視訊聊天,視訊美女

. said...

酒店經紀PRETTY GIRL 台北酒店經紀人 ,禮服店 酒店兼差PRETTY GIRL酒店公關 酒店小姐 彩色爆米花酒店兼職,酒店工作 彩色爆米花酒店經紀, 酒店上班,酒店工作 PRETTY GIRL酒店喝酒酒店上班 彩色爆米花台北酒店酒店小姐 PRETTY GIRL酒店上班酒店打工PRETTY GIRL酒店打工酒店經紀 彩色爆米花

酒店上班請找艾葳 said...

艾葳酒店經紀提供專業的酒店經紀,酒店上班,酒店打工、兼職、酒店相關知識等酒店相關產業服務,想加入這行業的水水們請找專業又有保障的艾葳酒店經紀公司!
艾葳酒店經紀是合法的公司、我們是不會跟水水簽任何的合約 ( 請放心 ),我們是不會強押水水辛苦工作的薪水,我們絕對不會對任何人公開水水的資料、工作環境高雅時尚,無業績壓力,無脫秀無喝酒壓力,高層次會員制客源,工作輕鬆。
一般的酒店經紀只會在水水們第一次上班和領薪水時出現而已,對水水們的上班安全一點保障都沒有!艾葳酒店經紀公司的水水們上班時全程媽咪作陪,不需擔心!只提供最優質的酒店上班環境、上班條件給水水們。

琪琪 said...

we have the highest quality but the lowest price fashion products wholesale from China.Here are the most popular China Wholesale products for all of you.You can visit
...................................................................

琪琪 said...

And if we are talking specifics, here, then make it a waist 33, length 30 to crease nicely over my square tipped Kenneth Cole Reaction vintage nubuck loafers. I also would like a 1.8" wide Gucci horsebit ring buckle belt the same color as the loafers.

............................................................................

琪琪 said...

Enjoy your shopping experience on mensclothingus.com.You can find the father who desire fashionable, intellectual mens clothing simultaneouslyGod bless you!I really agree with your opinions.
..............................................................

Anonymous said...

the loaf has been eaten by someone else. But life is real (not virtual) and this happens.
Dissertations | Research Papers | Term Papers

Anonymous said...

it's not just me! Statistical NLP took a long time to catch on and is still gaining in popularity.
Theses | Essays

naomi said...

情趣,情趣,情趣用品,情趣用品,情趣商品,情趣商品,按摩棒,跳蛋,情趣按摩棒,充氣娃娃,保險套,飛機杯,潤滑液,情趣內衣,性感內衣,g點,持久液,按摩棒,跳蛋,情趣按摩棒,充氣娃娃,保險套,飛機杯,潤滑液,情趣內衣,性感內衣,g點,持久液,按摩棒,跳蛋,情趣按摩棒,充氣娃娃,保險套,飛機杯,潤滑液,情趣內衣,性感內衣,g點,持久液,按摩棒,跳蛋,情趣按摩棒,充氣娃娃,保險套,飛機杯,潤滑液

Sheldon said...

like to try the personalized notebooks

Thesis Writing | Dissertation Writing | Essay Writing | Assignment Writing

tariely said...

Классный кино на кинозоуне.
электронная почта без регистрации

Sheldon said...

Great Post.. Thanks For Sharing

Thesis Help | Dissertation Help | Essay Help | Assignment Help

tar said...

электронная почта без регистрации

Anonymous said...

Фильмы онлайин

seldamuratim said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it to a few friends of mine that I know would enjoy reading..
sesli sohbetsesli chatkamerali sohbetseslisohbetsesli sohbet sitelerisesli chat siteleriseslichatsesli sohpetseslisohbet.comsesli chatsesli sohbetkamerali sohbetsesli chatsesli sohbetkamerali sohbet
seslisohbetsesli sohbetkamerali sohbetsesli chatsesli sohbetkamerali sohbet

speed dating new york said...

As a newbie, this article was really helpful, thanks for sharing!

daniel john said...

What an amazing post that I have ever come through. It gives the information that I was really searching for the past week and I am really satisfied with this post. Need more like this. Thank you.


Term papers

cilemsin42 said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it to a few friends of mine that I know would enjoy reading..
sesli sohbetsesli chat
sesli sohbet siteleri

sesli chat siteleri sesli sohbetsesli chat
sesli sohbet siteleri
sesli chat siteleri
SesliChat
cılgın sohbet
güzel kızlar
bekar kızlar
dul bayanlar
seviyeli insanlar
yarışma
canlı müzik
izdivac
en güzel evlilik
hersey burada
sesliparti
seslisohbet odalari
Sesli adresi
Sesli Chat
SesliChat Siteleri
Sesli Chat sitesi
SesliChat sitesi
SesliSohbet
Sesli Sohbet
Sesli Sohbet Sitesi
SesliSohbet Sitesi
SesliSohbet Siteleri
Muhabbet Sitesi
kamerali chat
Görüntülü Sohbet
Hasret gülleri
Çet sitesi
SesliSohbet
Sesli Sohbet
Canli sohbet
Turkce sohbet
Kurtce Sohbet
Kurtce Chat
Kurtce Muhabbet
Kurtce Sohbet
Kurdish Chat
SesliChat
Sesli Chat
SesliSanal
Guncel Haber
sohbet Sitesi
Chat sitesi..

Viko said...

Have a writing an essay task and do not actually know the way to start? Don’t be frustrated, just because the buy essay papers service will aid you with your online term paper very fast. Hence why not to use this kind of assistance?

seldamuratim said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it to a few friends of mine that I know would enjoy reading..

sesli sohbet
seslisohbet
sesli chat
seslichat
sesli sohbet sitesi
sesli chat sitesi
sesli sohpet
kamerali sohbet
kamerali chat
webcam sohbet

combattery84 said...

HP dv9700 battery
HP F4809A Battery
HP nc8000 battery
HP nc8230 battery
HP pavilion zd8000 battery
HP f2024b battery
HP f4812a battery
HP Pavilion ZV5000 battery
HP Pavilion DV1000 battery
HP Pavilion ZD7000 Battery
HP Pavilion DV2000 battery
HP Pavilion DV4000 Battery
HP Pavilion dv6000 Battery
HP Pavilion DV9000 Battery
HP F4098A battery
HP pavilion zx6000 battery
HP omnibook xe4400 battery
HP omnibook xe4500 battery
HP omnibook xe3 battery
Notebook NX9110 battery
IBM 02K6821 battery
IBM 02K7054 battery
IBM 08K8195 battery
IBM 08K8218 battery
IBM 92P1089 battery
IBM Thinkpad 390 Series battery
IBM Thinkpad 390X battery
IBM ThinkPad Z61m Battery
IBM 02K7018 Battery
IBM thinkpad t41p battery
IBM THINKPAD T42 Battery

combattery84 said...

IBM ThinkPad R60 Battery
IBM ThinkPad T60 Battery
IBM ThinkPad T41 Battery
IBM ThinkPad T43 Battery
IBM ThinkPad X40 Battery
Thinkpad x24 battery
ThinkPad G41 battery
IBM thinkpad r52 battery
Thinkpad x22 battery
IBM thinkpad t42 battery
IBM thinkpad r51 battery
Thinkpad r50 battery
IBM thinkpad r32 battery
Thinkpad x41 battery
SONY VGP-BPS2 Battery
SONY VGP-BPS2C Battery
SONY VGP-BPS5 battery
SONY VGP-BPL2C battery
SONY VGP-BPS2A battery
SONY VGP-BPS2B battery
SONY PCGA-BP1N battery
SONY PCGA-BP2E battery
SONY PCGA-BP2NX battery
SONY PCGA-BP2S battery
SONY PCGA-BP2SA battery
SONY PCGA-BP2T battery
SONY PCGA-BP2V battery
SONY PCGA-BP4V battery
SONY PCGA-BP71 battery
SONY PCGA-BP71A battery
SONY VGP-BPL1 battery
SONY VGP-BPL2 battery

DiSCo said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it

to a few friends of mine that I know would enjoy reading..
seslisohbet
seslichat
sesli sohbet
sesli chat
sesli
sesli site
görünlütü sohbet
görüntülü chat
kameralı sohbet
kameralı chat
sesli sohbet siteleri
sesli chat siteleri
görüntülü sohbet siteleri
görüntülü chat siteleri
kameralı sohbet siteleri
canlı sohbet
sesli muhabbet
görüntülü muhabbet
kameralı muhabbet
seslidunya
seslisehir
sesli sex

Sesli Chat said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it

to a few friends of mine that I know would enjoy reading..
seslisohbet
seslichat
sesli sohbet
sesli chat
sesli
sesli site
görünlütü sohbet
görüntülü chat
kameralı sohbet
kameralı chat
sesli sohbet siteleri
sesli chat siteleri
sesli muhabbet siteleri
görüntülü sohbet siteleri
görüntülü chat siteleri
görüntülü muhabbet siteleri
kameralı sohbet siteleri
kameralı chat siteleri
kameralı muhabbet siteleri
canlı sohbet
sesli muhabbet
görüntülü muhabbet
kameralı muhabbet
birsesver
birses
seslidunya
seslisehir
sesli sex