Mixed-Integer Optimal Control

Lecture 3 of IRC, LIAM, Fields-CQAM MfPH Training Series

Sanafi, NSERC, YorkU

Professor Michael Chen

2020-03-23

@York University

Mixed Integer Optimal Control Problem

Each of the following three topics:

has rich theories and abundant numerical algorithms.

Today we focus on their interactions.

Mixed Integer Optimal Control Problem

\begin{align*} &\underset{x(\cdot),u(\cdot),w(\cdot)}{\text{minimize}} & &\psi(x(\cdot),u(\cdot),w(\cdot)) \\ &\text{subject to} & &\dot{x}(t) = f(t, x(t),u(t), w(t)) \;\; \forall t \in \mathcal{T}, \\ &&&0 \le c(t, x(t),u(t),w(t)) \;\; \forall t \in \mathcal{T}, \tag{2.1}\\ &&&0 \le r(\{x(t_i)\}) \;\; \{t_i \}\subset \mathcal{T}, \\ \end{align*}

where codomain of the discrete control \(w(\cdot)\) is a finite set \(\Omega\) with \(n^{\Omega}\) values (\(n^{\Omega}<\infty\)); other details are the same as (1.1) in lecture 2 slides.

Direct Single Shooting

./images/fig1.1.png

Don't confuse the interity requirment with the discretization grid.

Direct Multiple Shooting

./images/fig1.2.png

Multiple Shooting Discretized MIOCP

\begin{align*} &\underset{s,q,w}{\text{minimize}} & &\sum_{i=1}^{m} l_i(t_i,s_i,q_i,w_i) \\ &\text{subject to} & &0 = s_{i+1} - x_i(t_{i+1};t_i,s_i,b_i(t_i,q_i),w_i) \;\; 0\le i \le m-1, \\ &&&0 \le c_i(t_i, s_i,b_i(t_i,q_i),w_i) \;\; 0\le i \le m, \\ &&&0 \le r_i(t_i,s_i,b_i(t_i,q_i),w_i) \;\; 0\le i \le m, \\ &&& w_i \in \{0,1\}^{n^w} \;\; (2.3) \end{align*}

It is an integer NLP.

Integer/Binary/Discretization Control Functions

For an integer control function \(0 \le \hat{v}(\cdot) < n^v:\)

\begin{equation*} \hat{v}(t) \equiv \sum_{i=0}^{\lceil \log_2 n^v \rceil}{2^i w_i(t)}, \end{equation*}

where \(w_i(t)\) is binary; for other discrete control functions:

\begin{equation*} f(t,x(t),u(t),w(t)) \equiv \sum_{i=1}^{n^v}{f(t,x(t),u(t),v_i) w_i(t)}, \end{equation*}

similarly for \(\psi(\cdot), c(\cdot)\).

Why MIOCP is harder?

Why MIOCP is harder?

About Relax

./images/relax.jpg

Better Relaxation: Jensen's inequality

./images/Jensen.png

Convexification and Relaxation

Tight bounds means better relaxation. We are able to get tight bounds on some problems.

Binary Nonlinear Problem (BN)

\begin{align*} &\underset{x(\cdot),u(\cdot),w(\cdot)}{\text{minimize}} & &\psi(x(\cdot),u(\cdot),w(\cdot)) \\ &\text{subject to} & &\dot{x}(t) = f(t, x(t),u(t), w(t)) \;\; \forall t \in \mathcal{T}, \\ &&& x(t_0) = x_0, \\ &&& w(t) \in \{0,1\}^{n^w} \;\; \forall t \in \mathcal{T} \end{align*}

Relaxed Nonlinear Problem (RN)

\begin{align*} &\underset{x(\cdot),u(\cdot),w(\cdot)}{\text{minimize}} & &\psi(x(\cdot),u(\cdot),w(\cdot)) \\ &\text{subject to} & &\dot{x}(t) = f(t, x(t),u(t), w(t)) \;\; \forall t \in \mathcal{T}, \\ &&& x(t_0) = x_0, \\ &&& w(t) \in [0,1]^{n^w} \;\; \forall t \in \mathcal{T} \end{align*}

Binary Convexified Linear Problem (BC)

\begin{align*} &\underset{x(\cdot),u(\cdot),w(\cdot)}{\text{minimize}} & &\sum_{i=1}^{2^{n^w}} \psi(x(\cdot),u(\cdot),w^i)w_i(\cdot) \\ &\text{subject to} & &\dot{x}(t) = \sum_{i=1}^{2^{n^w}}f(t, x(t),u(t), w^i)w_i(t) \;\; \forall t \in \mathcal{T}, \\ &&& x(t_0) = x_0, \\ &&& w(t) \in \{0,1\}^{n^w} \;\; \forall t \in \mathcal{T}\\ &&& 1 = \sum_{i=1}^{2^{n^w}}w_i(t). \end{align*}

Specicial Ordered Set One (SOS1)

\begin{equation*} 1 = \sum_{i=1}^{2^{n^w}}w_i(t), w(t) \in \{0,1\}^{n^w} \;\; \forall t \in \mathcal{T} \end{equation*}

SOS property is important for branching in solving integer problems.

Relaxed Convexified Linear Problem (RC)

\begin{align*} &\underset{x(\cdot),u(\cdot),w(\cdot)}{\text{minimize}} & & \sum_{i=1}^{2^{n^w}} \psi(x(\cdot),u(\cdot),w^i)\alpha_i(\cdot) \\ &\text{subject to} & &\dot{x}(t) = \sum_{i=1}^{2^{n^w}}f(t, x(t),u(t), w^i)\alpha_i(\cdot) \;\; \forall t \in \mathcal{T}, \\ &&& x(t_0) = x_0, \\ &&& \alpha(t) \in [0,1]^{n^w} \;\; \forall t \in \mathcal{T}\\ &&& 1 = \sum_{i=1}^{2^{n^w}}\alpha_i(t). \end{align*}

BN,RN,BC,RC

./images/fig2.2.png

Bounds in LOCP and Bang-Bang

Linear Control Problem:

\begin{align*} \dot{x}(t) &= A(t)x(t) + B(t) u(t) \;\; \forall t \in \mathcal{T}, \\ x(t_0) &= x_0 \\ u(t) &\in [u^{lb},u^{ub}]^{n^w} \;\; \forall t \in \mathcal{T} \;\;\;\;\; (2.6) \end{align*}

Bang-Bang controls:

\begin{equation*} \{u: \mathcal{T} \to \mathbb{R}^{n^u} | \forall t \in \mathcal{T} , 1 \le i \le n^u, u_i(t) \in \{u_i^{lb}, u_i^{ub}\} \} \end{equation*}

Conclusion: there is always an optimal bang-bang control; optimal set of both (RN) and (RC) cover the optimal set of (BN).

Bounds on Objective Functions of BN and BC

\begin{equation*} \psi^{BN} = \psi^{BC} \;\;\tag{Theorem 2.2} \end{equation*}

Bounds on Objective Functions of BC and RC

\begin{equation*} \exists w_{\epsilon} \in \{0,1\}^{n^w}, \psi_{\epsilon}^{BC} \le \psi^{RC} + \epsilon \;\;\tag{Theorem 2.3} \end{equation*}

Bounds on BN, RN, BC, RC and infeasibility

\begin{equation*} \psi^{RN} \le \psi^{RC} \le \psi_{\epsilon}^{BN} = \psi_{\epsilon}^{BC} \le \psi^{RC} + \epsilon \;\;\tag{Theorem 2.4} \end{equation*}

Thus the true \(\psi^{BN}\) could be arbitrarily approximated by a feasible solution close to the boundary of RC.

Again, BN does not have the path constraint. In case the path constraint does not involve the binary control, the bounds still apply; o.w., fail.

Example of Path Constraint with Integer Control

\begin{align*} 0 &\le 1 - 10^{-n} - w(t)\\ 0 &\le w(t) - 10^{-n}, n\ge 1 \end{align*}

It is infeasible but has a solution for a relaxed \(w(t)\in [0,1]\).

\begin{align*} 0 &<= (1 - 10^{-n})\alpha_1(t) - 10^{-n}\alpha_2(t) \\ 0 &<= -10^{-n} \alpha_1(t) + (1 - 10^{-n})\alpha_2(t) \end{align*}

The Outer Convexification of Constraint has no solution, which reveals the infeasibility of the original problem.

Rounding Strategies for RC

Suppose we have solved (RC) by multiple shooting and get \(\alpha^*(\cdot)\). In general \(\alpha^*(\cdot)\) is not binary, not even for the relaxed LOCP. A switching point of the convex multiplier \(\alpha^*(\cdot)\) may fall in-between an interval of the discretization grid of a multiple shooting algorithm, in this case \(\alpha^*(\cdot)\) will be fractional in that interval. Theorem 2.4 suggests us to find a feasible solution close to the boundary of (RC). We apply rounding huristics for this purpose. Note that for a linear (BN), its (RC) degenerates to (RN), i.e., the convexification is equivalent to the direct relaxation; for a nonlinear (BN), its (RC) has an extra SOS1 constraint.

Direct Rounding

For a linear (BN), let \(\alpha^*(\cdot)\) be the solution of (RC). Then the direct rounding solution \(w(t)=p_i\in \{0,1\}^{n^w}\) for \(t \in [t_i , t_{i+1}]\) on the grid \(\{t_i\}, 0 \le i \le m\) is defined by

\begin{equation*} p_{i,j} = 1 \;\;\; \text{ if } \int_{t_i}^{t_{i+1}} \alpha^*(t) dt \ge \frac{1}{2}; 0 \text{ otherwise.} \end{equation*}

Sum-up Rounding and Bounds

For a linear (BN),

\begin{equation*} p_{i,j} = 1 \;\;\; \text{ if } \int_{t_0}^{t_{i+1}} \alpha^*(t) dt - \sum_{k=0}^{k=i-1}p_{k,j} \delta t_k\ge \frac{1}{2} \delta t_i; 0 \text{ otherwise.} \end{equation*}
\begin{equation*} hence, \end{equation*}
\begin{equation*} \left|\int_{t_0}^{t} w(\tau) - \alpha(\tau) d\tau \right|_{\infty} \le \frac{1}{2} \max_i \delta t_i \end{equation*}

Sum-up rounding respects the accumulation effect of the control.

Sum-up Rounding and Bounds on State Trajectory

Let two initial value problems be given on the time horizon \(\tau = [t_0, t_f]\),

\begin{align*} \dot{x}(t) = A(t, x(t))\alpha(t), \forall t \in \tau , x(t_0) = x_0,\\ \dot{y}(t) = A(t, y(t))w(t), \forall t \in \tau , y(t_0) = y_0, \end{align*}

assume that \(\forall t \in \tau\) and almost everywhere,

\begin{align*} \|\frac{d}{dt}A(t,x(t))\| \le C \\ \|A(t,x(t)) - A(t,y(t))\| \le L\|x(t) - y(t)\| \\ \|A(t,x(t)\|_{\infty} \le M \\ \|\int_{t_0}^{t} w(\tau) - \alpha(\tau) d\tau \|_{\infty} \le \epsilon \end{align*}

Sum-up Rounding and Bounds of State Trajectory (Cont.)

then

\begin{equation*} \|y(t)-x(t)\| \le \left( \|y_0 - x_0\| +\left(M + C(t - t_0)\epsilon\right)\right)e^{L(t-t_0)} \end{equation*}

SOS1-Direct Rounding

Note that for a nonlinear (BN), its (RC) has an extra SOS1 constraint. For example, \(\sum_{i=1}^{100}x_i=1, x_i\in\{0,1\}\), then a relaxed solution hardly has any \(x_i>0.5\). We should boost the largest \(x_i\) to one. In case of a tie, we arbitrarily choose the \(x_i\) with the smallest index \(i\).

SOS1-SUR Rounding

\begin{align*} \hat{p}_{i,j} &= \int_{t_0}^{t_i} \alpha_j^*(t) dt - \sum_{k=0}^{i-1} p_{k,j}\\ p_{i,j} &= 1 \text{ if } \hat{p}_{i,j} \text{ is one of the largest with the smallest index,} \end{align*}

then

\begin{equation*} \left|\int_{t_0}^{t} w(\tau) - \alpha(\tau) d \tau \right|_{\infty} \le (n^w - 1) \max_i \delta t_i \end{equation*}

It is hard to obtain a bound on the state trajectory though.

Key Challenge

We will implement controls provided by (RC) + Rounding, not by (BN) or (BC). Hence we need to know the quality and characteristics of (RC) + Rounding.

Zeno's or chattering phenomenon

\begin{align*} &\underset{x(\cdot),u(\cdot),w(\cdot)}{\text{minimize}} & &\psi(x(\cdot),u(\cdot),w(\cdot)) \\ &\text{subject to} & &\dot{x}(t) = f(t, x(t),u(t), w(t)) \;\; \forall t \in \mathcal{T}, \tag{BN}\\ &&& x(t_0) = x_0, \\ &&& w(t) \in \{0,1\}^{n^w} \;\; \forall t \in \mathcal{T} \end{align*}

Oscillating \(w(\cdot)\) could be optimal, then what you will see in the \(\alpha^*(\cdot)\) of its (RC)?

Infinitely Many Switching in Finite Horizon

\begin{equation*} \psi^{RN} \le \psi^{RC} \le \psi_{\epsilon}^{BN} = \psi_{\epsilon}^{BC} \le \psi^{RC} + \epsilon \;\;\tag{Theorem 2.4} \end{equation*}

Beautiful bounds at a cost of infinitely many switching in a finite horizon.

Zeno's or chattering phenomenon

./images/Zeno.jpg

Zeno's or chattering phenomenon

./images/Zeno_Achilles_Paradox.png

Binary Convexified Linear Problem with Switching Cost (BCS)

\begin{align*} &\underset{x(\cdot),u(\cdot),w(\cdot)}{\text{minimize}} & & \sum_{i=1}^{2^{n^w}} \psi(x(\cdot),u(\cdot),w^i)\alpha_i(\cdot) + \sum_{j=1}^{n^w}\pi_j \sigma_j \\ &\text{subject to} & &\dot{x}(t) = \sum_{i=1}^{2^{n^w}}f(t, x(t),u(t), w^i)\alpha_i(\cdot) \;\; \forall t \in \mathcal{T}, \\ &&& x(t_0) = x_0,w(t) \in \{0,1\}^{n^w} \;\; \forall t \in \mathcal{T}, 1 = \sum_{i=1}^{2^{n^w}}w_i(t),\\ &&& \sigma_j=\sum_{i=0}^{m-1}|w_{i+1,j}-w_{i,j}|, \sigma_j \le \sigma_{j,max} \end{align*}

Relaxed Convexified Linear Problem with Switching Cost (RCS)

\begin{align*} &\underset{x(\cdot),u(\cdot),w(\cdot)}{\text{minimize}} & & \sum_{i=1}^{2^{n^w}} \psi(x(\cdot),u(\cdot),w^i)\alpha_i(\cdot) + \sum_{j=1}^{n^w}\pi_j \sigma_j \\ &\text{subject to} & &\dot{x}(t) = \sum_{i=1}^{2^{n^w}}f(t, x(t),u(t), w^i)\alpha_i(\cdot) \;\; \forall t \in \mathcal{T}, \\ &&& x(t_0) = x_0,w(t) \in [0,1]^{n^w} \;\; \forall t \in \mathcal{T}, 1 = \sum_{i=1}^{2^{n^w}}w_i(t),\\ &&& \sigma_j=\sum_{i=0}^{m-1}|w_{i+1,j}-w_{i,j}|, \sigma_j \le \sigma_{j,max} \end{align*}

Differentiable Reformulation

\begin{align*} \sigma_j &= \frac{1}{2}\sum_{i=0}^{m-1} \hat{\sigma}_{i,j}\\ \hat{\sigma}_{i,j} &\ge w_{i+1,j}-w_{i,j}\\ \hat{\sigma}_{i,j} &\ge w_{i,j}-w_{i+1,j} \end{align*}

Though correct for (BCS), it attracts fractional solutions in (RCS): \(w_{i+1,j}=w_{i,j}=\frac{1}{2}\), since it reduces the objective value of (RCS). After sum-up rounding, it becomes a chattering solution again. Put in another word, this reformulation leads to free switching, which directly defeats the purpose of switching cost.

Convex Reformulation

\begin{align*} \sigma_j &= \sum_{i=1}^{m-1} \sigma_{i,j}\\ \sigma_{i,j} &= \alpha_{i,j}(w_{i,j}+w_{i+1,j}) + \beta_{i,j}(2 - w_{i,j} - w_{i+1,j}) \\ 1 &= \alpha_{i,j} + \beta_{i,j} \end{align*}

No Free Switching, no chattering, no Zeno's Phenomenon

./images/tab2.1.png

Summary

Though it is possible to solve an integer OCP to optimality, we choose a more economical strategy and make best out of the limited computing budget.

Questions

./images/question.jpg

Bibliography

  1. KIRCHES, CHRISTIAN. Fast Numerical Methods for Mixed-Integer Nonlinear Model-Predictive Control, 2010, PhD dissertation.