To theory or not to theory

I stumbled upon this paper "Reflections after Refereeing Papers for NIPS" by Leo Breiman that gives some really candid insights into theory papers. (Unfortunately, I could not find a soft copy to share, except this link.) Some noteworthy observations:

"No theorems" implies "No theory"

"... more than 99% of the published papers are useless exercises."

"Mathematical theory is not critical to development of machine learning."

"Our fields would be better off with far fewer theorems, less emphasis on faddish stuff, and much more into scientific inquiry and engineering."

I really liked this article, especially coming from someone who has been working in theory all his life but I would still prefer reading papers giving theoretical insight, however useless, than pages and pages of feature engineering & experimentation using classifier X on problem Y -- the current trend at ACL.

NIPS papers are out

For a full list see here. Some papers I want to read based on my current interests:

Random Projections for Manifold Learning
Chinmay Hegde, Michael Wakin, Richard Baraniuk

The Distribution Family of Similarity Distances
Gertjan Burghouts, Arnold Smeulders, Jan-Mark Geusebroek

Manifold Sculpting
Michael Gashler, Dan Ventura, Tony Martinez

A learning framework for nearest neighbor search
Lawrence Cayton, Sanjoy Dasgupta

Learning Bounds for Domain Adaptation
John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, Jennifer Wortman

Convex Relaxations of EM
Yuhong Guo, Dale Schuurmans

A Randomized Algorithm for Large Scale Support Vector Learning
Krishnan Kumar, Chiru Bhattacharya, Ramesh Hariharan

Bundle Methods for Machine Learning
Alex Smola, S V N Vishwanathan, Quoc Le

Regularized Boost for Semi-Supervised Learning
Ke Chen, Shihai Wang

Learning the structure of manifolds using random projections
Yoav Freund, Sanjoy Dasgupta, Mayank Kabra, Nakul Verma

A complexity measure for intuitive theories
Charles Kemp, Noah Goodman, Joshua Tenenbaum