With Yann LeCun and Rob Fergus at the Computational and Biological Learning Lab at NYU, at NetScale Technologies in Northern NJ and with other Professors at the Courant Institute at NYU including Leslie Greengard and Paolo Barbano, I have worked on several projects which apply machine-learning techniques to real world data -- from Gigapixel images, to live video streams, to paint samples from the Metropolitan Museum to biological images of worms and mouse dendrites to 3D point-clouds of entire cities and robotics. I have worked in a collaboration between the Computer Science and Economics Departments at NYU and the NY Fed on real-estate transaction data. Most recently I have been working on a DARPA grant where we are seeking applications of Deep-Learning.
Deep-Learning, Gigapixel Photography, Situated video, Action Recognition
A DARPA program where we worked in collaboration with teams in Yoshua Bengio's lab at the University of Montreal and a group under Hans Peter Graf at NEC. We put together two demos to show applications of this exciting new technology.
Type "man with a red hat" or "woman running" this system parses natural language and maps phrases directly to the most relevant image patches in huge gigapixel images which we took from New York City rooftops.
Live 360 Video
Clement's system can parse each image in a 360 degree video stream into 12-30 classes (car,person,streetlight, etc.) in near real-time plus he can offload the heavy computing to his custom architecture on a low power FPGA chip.
Large Datasets, Ensemble Methods, Machine Learning
A project run by Andrew Caplin, Professor of Economics at NYU. We build a price surface which can predict the price of any house at any time. Currently we are only looking at LA County in a project about Mortgage defaults sponsored by the NY Fed
Large Datasets, 3D point-clouds, LIDAR, Sensor Fusion, Machine Learning
I was the main programmer for Net-Scale in the Net-Scale/HRL participation in the DARPA URGENT challenge. We built a stand-alone end-to-end system to which were input raw-point clouds and which produced polygonal classifications of large objects : trees, lawns, buildings, streets. The code was a C library which was linked from HRL's code base.
Autonomous Robotics, Vision, Machine Learning
Pierre Sermanet put together this video of some clips from the last few months which aims to explain most of the elements of the system which were working together in a pretty efficient end to end system
- Larger videos
- Yann LeCun's LAGR project page
- Raia Hadsell's page on LAGR and her collection of LAGR movies