The Next Generation of Neural Networks

Google Tech TalksNovember, 29 2007In the 1980's, new learning algorithms for neural networks promised tosolve difficult classification tasks, like speech or object recognition,by learning many layers of non-linear features. The results weredisappointing for two reasons: There was never enough labeled data tolearn millions of complicated features and the learning was much too slowin deep neural networks with many layers of features. These problems cannow be overcome by learning one layer of features at a time and bychanging the goal of learning. Instead of trying to predict the labels,the learning algorithm tries to create a generative model that producesdata which looks just like the unlabeled training data. These new neuralnetworks outperform other machine learning methods when labeled data isscarce but unlabeled data is plentiful. An application to very fastdocument retrieval will be described.Speaker: Geoffrey HintonGeoffrey Hinton received his BA in experimental psychology from Cambridge in1970 and his PhD in Artificial Intelligence from Edinburgh in 1978. He didpostdoctoral work at Sussex University and the University of California SanDiego and spent five years as a faculty member in the Computer Sciencedepartment at Carnegie-Mellon University. He then became a fellow of theCanadian Institute for Advanced Research and moved to the Department ofComputer Science at the University of Toronto. He spent three years from 1998until 2001 setting up the Gatsby Computational Neuroscience Unit at UniversityCollege London and then returned to the University of Toronto where he is aUniversity Professor. He holds a Canada Research Chair in Machine Learning. Heis the director of the program on "Neural Computation and Adaptive Perception"which is funded by the Canadian Institute for Advanced Research.Geoffrey Hinton is a fellow of the Royal Society, the Royal Society of Canada,and the Association for the Advancement of Artificial Intelligence. He is anhonorary foreign member of the American Academy of Arts and Sciences, and aformer president of the Cognitive Science Society. He received an honorarydoctorate from the University of Edinburgh in 2001. He was awarded the firstDavid E. Rumelhart prize (2001), the IJCAI award for research excellence(2005), the IEEE Neural Network Pioneer award (1998) and the ITAC/NSERC awardfor contributions to information technology (1992).A simple introduction to Geoffrey Hinton's research can be found in hisarticles in Scientific American in September 1992 and October 1993. Heinvestigates ways of using neural networks for learning, memory, perception andsymbol processing and has over 200 publications in these areas. He was one ofthe researchers who introduced the back-propagation algorithm that has beenwidely used for practical applications. His other contributions to neuralnetwork research include Boltzmann machines, distributed representations,time-delay neural nets, mixtures of experts, Helmholtz machines and products ofexperts. His current main interest is in unsupervised learning proceduresfor neural networks with rich sensory input.

Channel: People & Blogs
Uploaded: November 30, 1999 at 12:00 am
Author: googletechtalks

Length: 59:23
Rating: 4.88
Views: 54310

Tags: education  engedu  google  googletechtalks  talk  talks  techtalk  techtalks  

Video Comments