Editing Openai/693e3ce6-229c-8008-97dc-ab720cb1f95a (section)

==== The margin is ====

γ=min⁡i∈I⋆(λi−∣∇f(x⋆)i∣).\gamma = \min_{i\in I^\star}(\lambda_i - |\nabla f(x^\star)_i|).γ=i∈I⋆min(λi−∣∇f(x⋆)i∣).
Here the tightest inactive coordinate is node 2 (the only one with nonzero gradient), so

γ=λ2−∣∇f(x⋆)2∣=ρα2−α(1−α)(1−ρ)(1+α)2.\gamma = \lambda_2 - |\nabla f(x^\star)_2|
= \rho\alpha\sqrt{2} - \frac{\alpha(1-\alpha)(1-\rho)}{(1+\alpha)\sqrt{2}}.γ=λ2−∣∇f(x⋆)2∣=ρα2−(1+α)2α(1−α)(1−ρ).
This simplifies neatly to a linear function of ρ−ρ0\rho-\rho_0ρ−ρ0:

  γ=α(3+α)(1+α)2  (ρ−ρ0).  \boxed{\;
\gamma
= \frac{\alpha(3+\alpha)}{(1+\alpha)\sqrt{2}}\;(\rho-\rho_0).
\;}γ=(1+α)2α(3+α)(ρ−ρ0).
So:
* At ρ=ρ0\rho=\rho_0ρ=ρ0, γ=0\gamma=0γ=0.
* For ρ=ρ0+δ\rho=\rho_0+\deltaρ=ρ0+δ with tiny δ>0\delta>0δ>0, you get γ=Cδ\gamma = C\deltaγ=Cδ with C=α(3+α)(1+α)2C=\frac{\alpha(3+\alpha)}{(1+\alpha)\sqrt{2}}C=(1+α)2α(3+α).

Since ρ\sqrt{\rho}ρ is Θ(1)\Theta(1)Θ(1) here (because ρ0\rho_0ρ0 is a constant depending on α\alphaα), taking δ\deltaδ extremely small makes γ\gammaγ much smaller than ρ\sqrt{\rho}ρ.

===== Take α=0.15\alpha=0.15α=0.15. Then =====

ρ0=0.853.15≈0.26984,ρ0≈0.519.\rho_0 = \frac{0.85}{3.15}\approx 0.26984,
\qquad
\sqrt{\rho_0}\approx 0.519.ρ0=3.150.85≈0.26984,ρ0≈0.519.
Now choose ρ=ρ0+10−6\rho=\rho_0+10^{-6}ρ=ρ0+10−6. Then

γ≈0.15⋅3.151.15⋅1.414⋅10−6≈2.9×10−7≪0.519≈ρ.\gamma
\approx \frac{0.15\cdot 3.15}{1.15\cdot 1.414}\cdot 10^{-6}
\approx 2.9\times 10^{-7}
\ll 0.519 \approx \sqrt{\rho}.γ≈1.15⋅1.4140.15⋅3.15⋅10−6≈2.9×10−7≪0.519≈ρ.
So this path graph gives you a very explicit, closed-form “bad” case where

γ<ρ\gamma < \sqrt{\rho}γ<ρ
by an arbitrarily large margin, while the true support is tiny (∣S⋆∣=1≤1/ρ|S^\star|=1\le 1/\rho∣S⋆∣=1≤1/ρ).

===== Algorithm parameters (t,βt,\betat,β, restart, etc.) don’t change γ\gammaγ: γ\gammaγ is a property of the instance (graph + α,ρ,s\alpha,\rho,sα,ρ,s) at the true optimizer. What algorithm parameters affect is how much transient “support thrashing” you might see before you enter the γ\gammaγ-neighborhood. =====

If you want, I can also give a path-prefix support construction (where S⋆={1,2,…,k}S^\star=\{1,2,\dots,k\}S⋆={1,2,…,k}) and show the breakpoint for node k+1k+1k+1, so you can make γ\gammaγ tiny while ∣S⋆∣≈k≈1/ρ|S^\star|\approx k\approx 1/\rho∣S⋆∣≈k≈1/ρ.