Editing Openai/693e3ce6-229c-8008-97dc-ab720cb1f95a (section)

==== In an accelerated prox-gradient method, you compute ====

xk+1=prox⁡tg(yk−t∇f(yk)).x_{k+1}=\operatorname{prox}_{t g}(y_k-t\nabla f(y_k)).xk+1=proxtg(yk−t∇f(yk)).
Apply the lemma with y=yky=y_ky=yk:

∣A(yk)∣ ≤ ((1+tL) ∥yk−x⋆∥2tγ)2.|A(y_k)|\ \le\ \left(\frac{(1+tL)\,\|y_k-x^\star\|_2}{t\gamma}\right)^2.∣A(yk)∣ ≤ (tγ(1+tL)∥yk−x⋆∥2)2.
So you can bound the number of “wrong” active coordinates at iteration kkk as soon as you have any bound on ∥yk−x⋆∥2\|y_k-x^\star\|_2∥yk−x⋆∥2 (or on an objective gap, using strong convexity).

And the identification literature you’re implicitly leaning on (active-set / manifold identification) does exactly this: it bounds the “active set complexity” in terms of a margin-like parameter (often called δmin⁡\delta_{\min}δmin) and a convergence bound εx(k)\varepsilon_x(k)εx(k). Proceedings of Machine Learning Research<ref>{{cite web|title=Proceedings of Machine Learning Research|url=https://proceedings.mlr.press/v89/sun19a/sun19a.pdf|publisher=Proceedings of Machine Learning Research|access-date=2025-12-19}}</ref>