I usually think of incremental improvements as the result of taking an existing result X and twiddling X a little to yield Y. This can happen in system development (adding syntax to my MT system helped), theory (Y is a straightforward corollary of X), etc. Incremental improvements are often uninteresting, unless your research area happens to be exactly that of X (i.e., if I work on MT, maybe I will try adding syntax now). But in a broader sense, everything is an incremental improvement. It is vanishingly unlikely that a paper will come along that is not incremental in some sense.
I think what often ends up making a paper more or less successful is how many incremental improvements it contains. For instance, the original CRF paper was clearly successful. Yet, mathematically, CRFs are only incrementally different from MEMMs. Experimentally, they only perform slightly better. Algorithmically, the optimization is only slightly more clever than inference in HMMs. Theoretically, they can solve only a slightly broader family of problems. But importantly, all of these advantages come in a bundle. The CRF is a success largely because a single, relatively simple formalism made (incremental) improvements in at least four areas. I think you can make similar arguments about other "must reads" in NLP, such as Collins' original parsing paper.
If true, this might lead one to make research and accept/reject decisions on the basis of the number of areas in which improvements are made. This would certainly cut down on the number of papers published, but I also feel that many purely incremental papers do end up being useful, if only as steps in a path. For instance, MEMMs themselves are much more incremental upon HMMs, maximum entropy models and some older work by both Brill and Roth. The important question is whether CRFs would have been discovered, had MEMMs not been used and exhibited problems (eg., the label-bias problem). Of course, such observations are easy in retrospect: I don't know how to identify them without seeing the goal.
(Incremental improvements also serve a second role: they are advertisements for the original technique, for those who missed it the first time around.)
Parsing floats at over a gigabyte per second in C#
11 hours ago
3 comments:
There's a saying somewhere that goes like this: "The best research are the ones that seem straightforward in hindsight." In other words, great ideas are often simple yet powerful modifications of existing theory/algorithms/systems, and that might be why great ideas like CRFs appear like "incremental improvement".
Perhaps one can draw an analogy to the idea of punctuated equilibrium in evolutionary biology. The idea is that evolution isn't a slow, continuous process without sudden jumps. Instead, a small variation may be a tipping point in rapid speciation. It might be interesting to view the evolution of research ideas in this light.
well, i'm not saying that CRFs are obvious...i think there's a difference between an obvious improvement and an incremental one.
i think the punctuated equilibrium analogy is pretty good...i hadn't thought of it, but it makes complete sense. it's also a good argument for "high recall" conferences, since we're essentially holding our collective breaths until something punctuates through. i think there's a large open question as to how we can simultaneously do this and lessen the amount of stuff out there to read. (and if you think we have it bad, one of my friends who does medicine/bio stuff, studying alzheimer's, has an advisor who reads 3000 abstracts per month.)
酒店經紀PRETTY GIRL 台北酒店經紀人 ,禮服店 酒店兼差PRETTY GIRL酒店公關 酒店小姐 彩色爆米花酒店兼職,酒店工作 彩色爆米花酒店經紀, 酒店上班,酒店工作 PRETTY GIRL酒店喝酒酒店上班 彩色爆米花台北酒店酒店小姐 PRETTY GIRL酒店上班酒店打工PRETTY GIRL酒店打工酒店經紀 彩色爆米花
Post a Comment