Lab Home | Phone | Search
Center for Nonlinear Studies  Center for Nonlinear Studies
 Colloquia Archive 
 Postdoc Seminars Archive 
 Quantum Lunch 
 CMS Colloquia 
 Q-Mat Seminars 
 Q-Mat Seminars Archive 
 Kac Lectures 
 Dist. Quant. Lecture 
 Ulam Scholar 
 Summer Research 
 Past Visitors 
 History of CNLS 
 Maps, Directions 
 CNLS Office 
Wednesday, January 23, 2008
2:00 PM - 3:00 PM
CNLS Conference Room (TA-3, Bldg 1690)


Dynamic and Neuro-Dynamic Programming: An Overview and Recent Work

Dimitri P. Bertsekas
McAfee Professor of Engineering, Lab. for Information and Decision Systems, Massachusetts Institute of Technology

Dynamic programming is a broadly applicable methodology for sequential decision making, but suffers from exponential growth of computational requirements as the problem size increases. This has led to extensive work on approximations over the last twenty years. One key idea is to use an (approximate) scoring function to select decisions in complex dynamic systems, arising in a broad variety of applications from engineering design, operations research, resource allocation, finance, etc. This is much like what is done in computer chess and computer backgammon, where positions are evaluated by means of a scoring function and the move that leads to the position with the best score is chosen. Neuro-dynamic programming/reinforcement learning provides a class of systematic methods for computing appropriate scoring functions using neural network-like approximation schemes and simulation/evaluation of the system's performance. Another important idea is to use heuristics to compute on-line the values of an approximate scoring function, and is well-suited for large discrete optimization problems. The talk will overview these methodologies and discuss recent work.

Host: Frank Alexander