Cost function properties in Multi-Armed Bandits

I've been reading Section 1.3, Proposition 1.3.1. in "Dynamic Programming and Optimal Control, vol 2" by Dimitri Bertsekas, where author posed a proof of a proposition as an exercise to a reader. The proposition described properties of the optimal reward function $J(x,M)$, paraphrased here for reference:

Let $B=ma{x}_{l}ma{x}_{x^l}\mid R^l(x^l)\mid$. For fixed $x$, the optimal reward function $J(x,M)$ has the following properties as a function of M:

1. $J(x,M)$ is convex and monotinically nondecreasing.
2. $J(x,M)$ is constant for $M\le -B/(1-\alpha )$
3. $J(x,M)$ = $M$ for $M\ge B/(1-\alpha )$

Author proceeded with the proof by considering the reward function at timestep $k$: $J_k$ and using classical DP iteration: $J_{k+1}(x,M)=max(M,ma{x}_l{L}^l(x_l,M,J_k))$, where we choose between retiring and continuing working on projects. We know as $k\to \mathrm{\infty }$ $J_k\to J$. He proved the 1st property, and below I will prove the last two.

I will prove this by induction on $k$. Properties 2 and 3 are evident for $k=0$: $$J_0=max(0,M)=0$$ for $M<-B/(1-\alpha )$ and $$J_0=M$$ for $M>B/(1-\alpha )$.

Where $$B=ma{x}_{l}ma{x}_{x_l}\mid R(x_l)\mid$$. Essentially we want to bound $L^l$: $\bar{L}$ is an infinite geometric series: $B+\alpha B+{\alpha }^{2}B+...+{\alpha }^{n}B=B/(1-\alpha )$ is an upperbound for $L^l$.

Let's assume property 2 hold for some $k$. Particularly, $$J_k(x,M)=const$$ for $M\le -B/(1-\alpha )$. Then, $$\begin{array}{rl}{L}^l(x_k,M,J_k)& =R(x_k)+\alpha E\{J_k^l(x_1,..,f^l(x^l,w^l),x^{l+1},...,x_n,M)\}\\ & =R(x_k)+const\\ & =const\end{array}$$ , which is a constant in terms of $M$. Moreover, using an earlier bound, we know: $$\begin{array}{r}\mid L^l\mid \le B/(1-\alpha )\\ -B/(1-\alpha )\le L^l\le B/(1-\alpha )\end{array}$$ Therefore, $$\begin{array}{rl}{J}_{k+1}& =max(M,ma{x}_l{L}^l(x,J_k,M))\\ & =ma{x}_l{L}^l(\cdot )\\ & =const\end{array}$$, where the second equality follows from $-B/(1-\alpha )\le L^l$ and $M\le -B/(1-\alpha )$. Therefore, property 2 was proved for $k+1$ and it completes induction. Similar argument applies to property 3, except the last step where: $$\begin{array}{rlrl}{L}^l& \le B/(1-\alpha )& B/(1-\alpha )& \le M\end{array}$$ , so $J_{k+1}=M$.

Q.E.D.