Multi-Stage Optimal Decision Problems

Multi-stage optimal decision problems describe a wide class of control problems where decisions must be made in stages in order to minimize a certain total cost. The problem solving process is completed in stages, with each stage requiring a decision to be made, forming a decision sequence. Each decision sequence corresponds to an objective function value.

The optimal solution satisfies the Principle of Optimality

        A problem is said to satisfy the Principle of Optimality if the subsolutions of an optimal solution of the problem are themseleves optimal solutions for their subproblems.

        Therefore, the optimal solution of the problem can be found in the optimal solution of its subproblems. This determines the computation process: first, divide the problem into subproblems, conquer the optimal solution of the subproblems, and then combine the optimal solution of the original problem.

Subproblems overlap

        There are many overlapping sub problems in  the original problem. And the same subproblem is only solved once, so its efficiency is often higher than the enumeration method. The more sub problems are overlapped, the higher their efficiency.

          Dynamic programming is an algorithm design paradigm that provides effective and elegant solutions to a wide class of problems. The basic idea is to recursively divide a complex problem into a number of simpler subproblems; store the solutions to each of these subproblems; and, ultimately, use the stored answers to solve the original problem. By caching solutions to subproblems, dynamic programming can sometimes avoid exponential waste.

 Steps of Dynamic Programming

1. Prove that the Principle of Optimality holds.

          In fact, it means that the optimal solution of the larger problem can be found from the optimal solution of the subproblem. In this, all subproblems are needed to be solved in advance and then used to build up a solution to the larger problem.

2. Develop a recurrence relation that relates a solution to its subsolutions.

      Indicate what the initial values are for that recurrenec relation, and which term signifies the final solution. This is the heart of the design process.

3. The solution of the subproblem is combined in a bottom of manner to obtain the optimal solution of a given problem.