With Yann LeCun and Rob Fergus at the Computational and Biological Learning Lab at NYU, at NetScale Technologies in Northern NJ and with other Professors at the Courant Institute at NYU including Leslie Greengard and Paolo Barbano, I have worked on several projects which apply machine-learning techniques to real world data -- from Gigapixel images, to live video streams, to paint samples from the Metropolitan Museum to biological images of worms and mouse dendrites to 3D point-clouds of entire cities and robotics. I have worked in a collaboration between the Computer Science and Economics Departments at NYU and the NY Fed on real-estate transaction data. Most recently I have been working on a DARPA grant where we are seeking applications of Deep-Learning.


DARPA Deep-Learning

Deep-Learning, Gigapixel Photography, Situated video, Action Recognition

@ NYU

A DARPA program where we worked in collaboration with teams in Yoshua Bengio's lab at the University of Montreal and a group under Hans Peter Graf at NEC. We put together two demos to show applications of this exciting new technology.

Gigapixel Images

Type "man with a red hat" or "woman running" this system parses natural language and maps phrases directly to the most relevant image patches in huge gigapixel images which we took from New York City rooftops.

Live 360 Video

Clement's system can parse each image in a 360 degree video stream into 12-30 classes (car,person,streetlight, etc.) in near real-time plus he can offload the heavy computing to his custom architecture on a low power FPGA chip.

Collaborators

Yann LeCun, Rob Fergus, Yoshua Bengio, Urs Muller, Clement Farabet, Pierre Sermanet, Koray Kavukcuoglu, Li Wan, Antoine Bordes, Hugo Larochelle, Aaron Courville, Pascal Lamblin


Real-Estate Price Prediction

Large Datasets, Ensemble Methods, Machine Learning

@ NYU

A project run by Andrew Caplin, Professor of Economics at NYU. We build a price surface which can predict the price of any house at any time. Currently we are only looking at LA County in a project about Mortgage defaults sponsored by the NY Fed

Collaborators

Andrew Caplin, Yann Lecun, Sumit Chopra, Roy Lowrance,


DARPA URGENT

Large Datasets, 3D point-clouds, LIDAR, Sensor Fusion, Machine Learning

@ Net-Scale Technologies

I was the main programmer for Net-Scale in the Net-Scale/HRL participation in the DARPA URGENT challenge. We built a stand-alone end-to-end system to which were input raw-point clouds and which produced polygonal classifications of large objects : trees, lawns, buildings, streets. The code was a C library which was linked from HRL's code base.

Collaborators

Yann Lecun, Urs Muller


LAGR Program :: Videos

Autonomous Robotics, Vision, Machine Learning

@ Net-Scale/NYU

Pierre Sermanet put together this video of some clips from the last few months which aims to explain most of the elements of the system which were working together in a pretty efficient end to end system

More Info

Collaborators

Yann Lecun, Urs Muller, Pierre Sermanet, Raia Hadsell, Matt Grimes, Ayshe,


Biological Imaging

Biological Imaging, Vision, Machine Learning

@ NYU

Classified spine vs. dendrite. Collaboration initiated by Paolo Barbano with Paul Greengard's Lab at Rockefeller University

Labeled image for worm vs. egg vs. background classification. Collaboration initiated by Paolo Barbano with Fabio Piano's Lab at NYU

Collaborators

Yann Lecun, Paolo Barbano, Leslie Greengard, Fabio Piano

CC


All parts of this website are licensed under a Creative Commons License.