Nonlinear Optimization

by Andrzej Ruszczynski

Overview

Optimization is one of the most important areas of modern applied mathematics, with applications in fields from engineering and economics to finance, statistics, management science, and medicine. While many books have addressed its various aspects, Nonlinear Optimization is the first comprehensive treatment that will allow graduate students and researchers to understand its modern ideas, principles, and methods within a reasonable time, but without sacrificing mathematical precision. Andrzej Ruszczynski, a leading expert in the optimization of nonlinear stochastic systems, integrates the theory and the methods of nonlinear optimization in a unified, clear, and mathematically rigorous fashion, with detailed and easy-to-follow proofs illustrated by numerous examples and figures.


The book covers convex analysis, the theory of optimality conditions, duality theory, and numerical methods for solving unconstrained and constrained optimization problems. It addresses not only classical material but also modern topics such as optimality conditions and numerical methods for problems involving nondifferentiable functions, semidefinite programming, metric regularity and stability theory of set-constrained systems, and sensitivity analysis of optimization problems.


Based on a decade's worth of notes the author compiled in successfully teaching the subject, this book will help readers to understand the mathematical foundations of the modern theory and methods of nonlinear optimization and to analyze new problems, develop optimality theory for them, and choose or construct numerical solution methods. It is a must for anyone seriously interested in optimization.

Product Details

ISBN-13:	9781400841059
Publisher:	Princeton University Press
Publication date:	09/19/2011
Sold by:	Barnes & Noble
Format:	eBook
Pages:	464
File size:	26 MB
Note:	This product may take a few minutes to download.

About the Author

Andrzej Ruszczynski is Professor of Operations Research at Rutgers University. He is the coauthor of Stochastic Programming and the coeditor of Decision Making under Uncertainty.

Table of Contents

Preface xi

Chapter 1. Introduction 1


PART 1. THEORY 15


Chapter 2. Elements of Convex Analysis 17
2.1 Convex Sets 17
2.2 Cones 25
2.3 Extreme Points 39
2.4 Convex Functions 44
2.5 Subdifferential Calculus 57
2.6 Conjugate Duality 75


Chapter 3. Optimality Conditions 88
3.1 Unconstrained Minima of Differentiable Functions 88
3.2 Unconstrained Minima of Convex Functions 92
3.3 Tangent Cones 98
3.4 Optimality Conditions for Smooth Problems 113
3.5 Optimality Conditions for Convex Problems 125
3.6 Optimality Conditions for Smooth-Convex Problems 133
3.7 Second Order Optimality Conditions 139
3.8 Sensitivity 150


Chapter 4. Lagrangian Duality 160
4.1 The Dual Problem 160
4.2 Duality Relations 166
4.3 Conic Programming 175
4.4 Decomposition 180
4.5 Convex Relaxation of Nonconvex Problems 186
4.6 The Optimal Value Function 191
4.7 The Augmented Lagrangian 196


PART 2. METHODS 209


Chapter 5. Unconstrained Optimization of Differentiable Functions 211
5.1 Introduction to Iterative Algorithms 211
5.2 Line Search 213
5.3 The Method of Steepest Descent 218
5.4 Newton's Method 233
5.5 The Conjugate Gradient Method 240
5.6 Quasi-Newton Methods 257
5.7 Trust Region Methods 266
5.8 Nongradient Methods 275


Chapter 6. Constrained Optimization of Differentiable Functions 286
6.1 Feasible Point Methods 286
6.2 Penalty Methods 297
6.3 The Basic Dual Method 308
6.4 The Augmented Lagrangian Method 311
6.5 Newton's Method 324
6.6 Barrier Methods 331


Chapter 7. Nondifferentiable Optimization 343
7.1 The Subgradient Method 343
7.2 The Cutting Plane Method 357
7.3 The Proximal Point Method 366
7.4 The Bundle Method 372
7.5 The Trust Region Method 384
7.6 Constrained Problems 389
7.7 Composite Optimization 397
7.8 Nonconvex Constraints 406


Appendix A. Stability of Set-Constrained Systems 411
A.1 Linear-Conic Systems 411
A.2 Set-Constrained Linear Systems 415
A.3 Set-Constrained Nonlinear Systems 418
Further Reading 427


Bibliography 431
Index 445

What People are Saying About This

Sachs

This excellent book is the best I have reviewed in the past ten years. Very well written, its three main strengths are its treatment of theory and algorithms on equal terms, its mathematically driven presentation of the material, and its interesting examples and applications.
— Ekkehard W. Sachs, Virginia Tech and Universitat Trier

Vanderbei

Nonlinear Optimization will become the standard textbook on its subject, as well as a reference book that everyone will want to own. Not only is it beautiful and elegant, it is also utterly comprehensive and modern, with many realistic and interesting examples.
— Robert J. Vanderbei, Princeton University, author of "Linear Programming"

From the Publisher

"Nonlinear Optimization will become the standard textbook on its subject, as well as a reference book that everyone will want to own. Not only is it beautiful and elegant, it is also utterly comprehensive and modern, with many realistic and interesting examples."—Robert J. Vanderbei, Princeton University, author of Linear Programming

"This excellent book is the best I have reviewed in the past ten years. Very well written, its three main strengths are its treatment of theory and algorithms on equal terms, its mathematically driven presentation of the material, and its interesting examples and applications."—Ekkehard W. Sachs, Virginia Tech and Universität Trier

Editorial Reviews

"This book offers a very good introduction to differentiable and nondifferentiable nonlinear optimization theory and methods. With no doubt the major strength of this book is the clear and intuitive structure and systematic style of presentation. This book can be recommended as a material for both self study and teaching purposes, but because of its rigorous style it works also as a valuable reference for research purposes."

Mathematical Modeling and Operational Research

"This is one of the best textbooks on nonlinear optimization I know. Focus is on both theory and algorithmic solution of convex as well as of differentiable programming problems."—-Stephan Dempe, Zentralblatt MATH Database

"In summary, this book competes with the topmost league of books on optimization. The wide range of topics covered and the thorough theoretical treatment of algorithms make it not only a good prospective textbook, but even more a reference text (which I am happy to have on my shelf.)"—-Franz Rendl, Operations Research Letters

"Throughout the book the writing style is very clear, compact and easy to follow, but at the same time mathematically rigorous. The proofs are easy to follow because the author usually carefully explains every move. In addition the meaning of the most central results is usually demonstrated with examples and in many cases explanations are also supported by visualizations...This book offers a very good introduction to differentiable and nondifferentiable nonlinear optimization theory and methods...Recommended as a material for both self study and teaching purposes"—-Petri Eskelinen, Mathematical Methods of Operation Research

From the Publisher

Throughout the book the writing style is very clear, compact and easy to follow, but at the same time mathematically rigorous. The proofs are easy to follow because the author usually carefully explains every move. In addition the meaning of the most central results is usually demonstrated with examples and in many cases explanations are also supported by visualizations...This book offers a very good introduction to differentiable and nondifferentiable nonlinear optimization theory and methods...Recommended as a material for both self study and teaching purposes

Mathematical Methods of Operation Research - Petri Eskelinen

In summary, this book competes with the topmost league of books on optimization. The wide range of topics covered and the thorough theoretical treatment of algorithms make it not only a good prospective textbook, but even more a reference text (which I am happy to have on my shelf.)

Operations Research Letters - Franz Rendl

This is one of the best textbooks on nonlinear optimization I know. Focus is on both theory and algorithmic solution of convex as well as of differentiable programming problems.

Zentralblatt MATH Database - Stephan Dempe

Throughout the book the writing style is very clear, compact and easy to follow, but at the same time mathematically rigorous. The proofs are easy to follow because the author usually carefully explains every move. In addition the meaning of the most central results is usually demonstrated with examples and in many cases explanations are also supported by visualizations...This book offers a very good introduction to differentiable and nondifferentiable nonlinear optimization theory and methods...Recommended as a material for both self study and teaching purposes
— Petri Eskelinen

Mathematical Methods of Operation Research

In summary, this book competes with the topmost league of books on optimization. The wide range of topics covered and the thorough theoretical treatment of algorithms make it not only a good prospective textbook, but even more a reference text (which I am happy to have on my shelf.)
— Franz Rendl

Operations Research Letters

This is one of the best textbooks on nonlinear optimization I know. Focus is on both theory and algorithmic solution of convex as well as of differentiable programming problems.
— Stephan Dempe

Zentralblatt MATH Database

This is one of the best textbooks on nonlinear optimization I know. Focus is on both theory and algorithmic solution of convex as well as of differentiable programming problems.
— Stephan Dempe

Zentralblatt MATH Database Database