Module 16
Optimisation algorithms
How a search finds a good design, why the answer depends on where it started, and the result that says no method wins everywhere.
What this module covers
- Describe a design space and say what makes one hard to search
- Distinguish a local optimum from a global one, and say why that distinction is expensive
- Compare deterministic and stochastic methods on equal evaluation budgets
- Explain exploration and exploitation as a trade-off rather than a preference
- Choose a stopping criterion and say what it gives up
- State the No Free Lunch result and what follows from it in practice
Lessons
- 16.1
The design space
What you are searching, why it is bigger than it looks, and the difference between a local and a global optimum.
Start lesson → Downhill methods, methods that sometimes go uphill, and why the second kind exists at all.
Start lesson →Why there is no best algorithm, and why every stopping criterion is a decision about what you are willing to miss.
Start lesson →
Module checkpoint
Check what you have taken in
3 questions
Question 1
A gradient method finds a better answer than a genetic algorithm on a smooth landscape, using a fifth of the evaluations. What follows?
Question 2
A design has 6 variables with 12 candidate values each. How many evaluations would a full grid search require?
Question 3
Simulated annealing proposes a move 1.2 units worse at a temperature of 0.4. What is the acceptance probability, to four decimal places?