AI Stats post mortem

Got back from AIStats a week or so ago; overall, I enjoyed it. Though by the end, I was totally sick of Puerto Rican food. (I'm a huge fan of Cuban food and expected stronger similarities. Oh well.) Here are some things I learned at AIStats:

• Multiple instance learning. There was a paper on this topic. I can't comment on how good the paper was (in comparison to other MIL papers), but I didn't know about the MIL task before. The problem is as follows. Instead of getting a set of labeled training points, you get a set of sets of points. Each set of points comes with a +1/-1 label. +1 means that at least one example in the set is positive; -1 means none are. Vision people like this because they have an image that contains a person but they don't know where the person is, so the scan a box around the image and generate lots of examples. We know at least one will contain a person. I realized after seeing the poster that I've actually worked on this problem! The BayeSum paper is actually a MIL algorithm, but I didn't know the right terminology. I wonder how it compares in vision problems.
• Deep belief networks. These seem to be growing in popularity -- both Hinton's group and LeCun's group have their own version of DBNs. You can basically think of these as neural nets with tons of layers of hidden units. They can presumably learn really complicated functions of inputs. It's also possible to train them unsupervisedly (is that a word?) and then just tune the unsupervised model to some classification task. I don't really understand how they work, but they seem to be doing something interesting.
• It is possible to learn (probably approximate) underestimates for the heuristic in A* search. The problem is that you have a given, fixed path cost and you want to learn an admissible heuristic. Seems quite similar to Berkeley paper at NAACL this year, though different in important ways. Seems like a reasonable thing to try if you're stuck with a fixed path cost. I'm curious how these techniques compare to keeping a heuristic fixed and learning a path cost, ala Drew -- seems that on the surface they'd be similar, but learning the path cost seems a bit easier to me.

There were, of course, other interesting papers. Yee Whye has listed a few.