Textbooks about optimization

This is a book on linear optimization, used for graduate-level courses at the University of Michigan. The examples are written using Python/Gurobi.

Topics include:

• Modeling.
• Algebra versus geometry.
• The simplex algorithm.
• Duality.
• Sensitivity analysis.
• Large-scale linear optimization.
• Integer-linear optimization.

Textbook: A first course in linear optimization.

• Python
• Gurobi
• Textbooks

This book provides a broad introduction to optimization with a focus on practical algorithms for the design of engineering systems. The text is intended for advanced undergraduates and graduate students as well as professionals. The examples are implemented in the Julia programming language.

Topics include:

• Derivatives and gradients.
• Stochastic methods.
• Constrained optimization.
• Sampling plans.
• Optimization under uncertainty.
• Multidisciplinary design optimization.

Textbook: Algorithms for optimization.

• Textbooks
• Julia

This online textbook, written by Stephen Boyd and Lieven Vandenberghe, covers many aspects of convex optimization.

According to the authors:

This book is about convex optimization, a special class of mathematical optimization problems, which includes least-squares and linear programming problems. It is well known that least-squares and linear programming problems have a fairly complete theory, arise in a variety of applications, and can be solved numerically very efficiently. The basic point of this book is that the same can be said for the larger class of convex optimization problems.

Boyd & Vandenberghe, preface of "Convex optimization"

This textbook includes many examples, along with Python, Julia, and Matlab data for additional exercises available on GitHub.

Textbook: Convex optimization.

GitHub: Data for additional exercises.

• Python
• Textbooks
• Julia
• Matlab

This is an online repository of companion notebooks for a project called "Data-Driven Mathematical Optimization in Python". It isn't exactly a textbook, but it is a great resource for learning about optimization modelling using the Pyomo library.

According to the authors, the goals of the notebooks are to:

• Provide a foundation for learning mathematical optimization hands-on.
• Demonstrate the tools and concepts of optimization with practical examples.
• Help others get started with building models and solving problems using a state-of-the-art modeling language and solvers.

The website is organized into chapters, with plenty of explanation and examples. There are around 50 notebooks, plus numerous supporting files, in the GitHub repository.

Textbook: Data-driven mathematical optimization in Python.

GitHub: Jupyter notebooks.

• Python
• Pyomo
• Textbooks

This e-book was written as a text for the topic "Decision Modeling" taught as a Business Management Course by David M. Tulett. There are numerous examples implemented using Excel Solver and LINGO.

Topics include:

• Applications of linear models.
• Sensitivity analysis.
• Network models.
• Integer models.
• Goal programming and nonlinear models.
• Decision analysis.

Textbook: Decision modeling.

• Solver
• Textbooks
• LINGO

This is a graduate-level textbook covering a range of fundamental to advanced optimization theory and algorithms with practical tips, numerous illustrations, and engineering examples.

Table of contents:

• Introduction.
• A short history of optimization.
• Numerical models and solvers.
• Unconstrained gradient-based optimization.
• Constrained gradient-based optimization.
• Computing derivatives.
• Gradient-free optimization.
• Discrete optimization.
• Multiobjective optimization.
• Surrogate-based optimization.
• Convex optimization.
• Optimization under uncertainty.
• Multidisciplinary design optimization.

Textbook: Engineering design optimization.

• Python
• Pyomo
• Textbooks

This is an open textbook from TU Delft University of Technology. It provides an overview of several Operations Research (OR) models that have a strong relevance for engineering problems and can be the basis for other extensions.

The book is designed to guide readers through a journey that starts with basic mathematical foundations, continues to the modeling recipe to properly define mathematical models and how to solve them, and finishes with a broad overview of models that reflect real-world operations.

Code for the examples is available in a GitHub respository. The code uses the Pyomo and gurobipy Python libraries.

Sections:

• Part I. Introduces the concepts of OR and serious games and gamification and justifies why such concepts were embedded into this education tool.
• Part II. Provides readers with a recap on vector and matrix notation, which is key to the mathematical modeling covered in this book, and with an overview of the ingredients of a mathematical model and of how to set up one.
• Part III. Builds on the previous one and covers the main solution methods that can be employed to solve a mathematical model.
• Part IV. Focuses on a first set of mathematical models, i.e., assignment problems, whose main goal is to determine how to properly assign items to resources.
• Part V. Focuses on network problems, where the goal is to efficiently route resources in a pre-defined environment (the network).
• Part VI. Presents an example of a modeling framework that accounts for uncertainty.

Textbook: From theORy to application: learning to optimize with Operations Research in an interactive way.

GitHub: Respository.

• Python
• Pyomo
• Textbooks

This online textbook, written by Changhyun Kwon, covers many aspects of optimization modelling in the Julia programming language using the JuMP (Julia for Mathematical Programming) package.

According to the author:

This book is neither a textbook in numerical methods, a comprehensive introductory book to Julia programming, a textbook on numerical optimization, a complete manual of optimization solvers, nor an introductory book to computational science and engineering—it is a little bit of all.

Changhyun Kwon, preface of "Julia programming for Operations Research"

This textbook includes many examples, which have source code available on GitHub.

Textbook: Julia programming for Operations Research.

GitHub: Code repository for textbook.

• Textbooks
• Julia

This book is a collection of papers about the art of mathematical modeling, by Tony Hürlimann. The book is for all persons eager to learn mathematical modeling, especially for students in operations research and management sciences or business analytics, in order to solve all kind of problems.

There are numerous examples, many of which can be run interactively via the author's website. The examples use the Linear Programming Language (LPL) modelling language, developed by the author. Although originally designed for linear problems, the language has expanded to cover a much wider range of problem types.

Sections:

• What is modeling? Gives an informal and a formal introduction to the concepts of modeling and model.
• How to model? A short introduction to the modeling process, explaining its stages in order to learn how to model.
• The modeling language LPL. An introduction to the LPL modeling language, highlighting features such as modularity, the drawing library, logical constraint, and goal programming.
• Index notation in mathematics. Explains indexing notation and its implementation in the modeling language LPL.
• Various modeling tools. Give a first impression of various modeling tools – free and commercial – that could be used to build mathematical models.
• Various model types. Discusses mathematical model classifications such as: discrete, continuous, linear, non-linear, differential equations, stochastic models, and fuzzy models.
• Data cubes and pivot-tables. Gives a survey and implementation in LPL on datacube and pivot-tables in order to understand and use OLAP functionalities.
• Logical modeling. Presents modeling techniques for formulating logical conditions in mathematical models.
• My vision. Explains the author's ideas of why we need a new programming language paradigm.

Textbook: Mathematical modeling basics.

• Python
• Textbooks

This is an open textbook in modeling, algorithms, and complexity. The many examples are written in Excel and Python (primarily the PuLP library).

The textbook is a work in progress.

Topics include:

• Linear programming.
• Integer programming.
• Discrete algorithms.
• Nonlinear programming.

Textbook: Mathematical programming and operations research.

• Solver
• Python
• Textbooks

This downloadable textbook, written by Richard Lusby & Thomas Stidsen, primarily teaches mathematical programming modelling and how to use Julia/JuMP for the implementation.

According to the authors:

In this book, we focus on how to create LP and MIP models. The models are first formulated in terms of mathematical equations, which are then implemented in the programming language Julia, using the JuMP package.

The implementation of a mathematical model using the JuMP package closely resembles how the mathematical model would be written. This makes the translation from the formulation to the implementation very intuitive.

Lusby & Stidsen, introduction of "Mathematical Programming with Julia"

This textbook includes many worked examples with Julia source code.

Textbook and source code: Mathematical Programming with Julia.

• Textbooks
• Julia

This online book is written by Eric Fraga at University College London. It is primarily intended to demonstrate that simple optimization methods, inspired by nature, are able to solve complex problems in process engineering. The book has examples written in the Julia programming language, with the source code available on GitHub.

According to the author:

The focus of this book is on solving problems that arise in process systems engineering (PSE). Optimization plays a crucial part in many PSE activities, including process design and operation. The problems that arise may have one or more of these challenges, in no particular order:

• Nonlinear models.
• Multi-modal objective function.
• Nonsmooth and discontinuous.
• Distributed quantities.
• Differential equations.
• Small feasible regions.
• Multiple objectives.

Fraga, introduction of "Nature inspired methods for optimization"

Book: Nature inspired methods for optimization.

GitHub: Code repository for textbook.

• Textbooks
• Julia

This online textbook, written by Fabio Schoen, is based on an advanced optimization modeling course for students in Management Engineering at Università degli Studi di Firenze.

The textbook's emphasis is on understanding how to model a variety of situations. According to the author:

I feel there is still a wide gap which separates academic research from day to day adoption. And, in my opinion, part of this is due to the lack of sufficient modeling skills. If we cannot model a problem, no matter how sophisticated our optimization algorithm are, we will not be able to solve it nor suggest any decision. We need to communicate our problem to a solver. And to do this we need a deep knowledge of models.

Fabio Schoen, preface of "Optimization models"

Consequently, the textbook describes many optimization model examples, with code written mostly in AMPL. Some examples also have Julia and Pyomo code, with the author welcoming contributions of new or existing models from readers.

Textbook: Optimization models.

• Pyomo
• Textbooks
• Julia
• AMPL

This is a two-part textbook about Column Generation: advanced Branch-Cut-and-Price algorithms.

Column generation is a method to solve linear programming problems with a very large number of variables. It dynamically generates variables (and the corresponding matrix columns, hence the name) by solving auxiliary optimization problems known as pricing subproblems. Column generation can also be very effective in integer programming, forming the backbone of algorithms like Branch-and-Price and Branch-Cut-and-Price.

Part I, Column generation basics:

1. The revised simplex algorithm
2. Dantzig-Wolfe decomposition and column generation for linear programming.
3. Integer programming review.
4. Dantzig-Wolfe decomposition and column generation for integer programming.
5. Lagrangian relaxation and column generation.

Part II, Topics in column generation:

6. Column generation based heuristics.
7. Column generation convergence.
8. Advanced branching.
9. The dynamic programming labeling algorithm for the RCSP.
10. Non-robust cuts.
11. Reduced cost fixing and related techniques.
12. Additional case studies.
13. Software for column generation.

Website: Optimizing with column generation.

• Textbooks

Optimization 101 is an online textbook written by Distinguished Research Professor Emeritus John W. Chinneck.

Topics include:

• Linear programming.
• Networks.
• Integer programming.
• Heuristics.
• Dynamic programming.
• Non-linear programming.

Textbook: Practical optimization: A gentle introduction.

• Textbooks