Editing Openai/694057b6-101c-8007-9a65-c40578c7252d (section)

==== This appendix defines a concrete, lightweight Linux liveness probe suitable for very long prefill. ====

===== A probe yields “evidence of life” if: =====
* the subprocess PID exists and is alive, and
* the process CPU time has increased since the last probe

CPU time increase indicates the process is doing work even if no tokens/bytes have been emitted yet.

===== Linux /proc/<pid>/stat contains: =====
* utime (user mode CPU ticks)
* stime (kernel mode CPU ticks)

A simple probe can parse the file and extract utime and stime as integers, then compute:
* cpu_ticks = utime + stime

Implementation notes
* Field parsing: /proc/<pid>/stat has a tricky second field (comm) which may contain spaces; parsing must account for the closing ) before splitting remaining fields.
* Don’t over-probe: interval default around 5s is fine.

===== Maintain per-worker probe state: =====
* last observed cpu_ticks (int)
* last probe time (monotonic timestamp)

On each probe:
# If PID no longer exists: - report “no liveness” (and process death likely triggers restart separately)
# Read current cpu_ticks
# If cpu_ticks > last_cpu_ticks: - update last_cpu_ticks - set last_liveness_at = now - return “alive with progress”
# Else: - return “alive but no cpu progress”

===== - During prefill (no stream bytes yet): - update last_liveness_at on positive CPU tick deltas - stall decision uses last_progress_at = max(last_liveness_at, last_stream_byte_at) =====

===== - Reading one small /proc file every few seconds is very cheap. =====
* If /proc read fails transiently: - treat as “no liveness update this time”, not an immediate failure - let stall policy decide over a longer window