Editing Openai/694057b6-101c-8007-9a65-c40578c7252d (section)

=== Assistant: Great — then we can simplify the request store a bit. ===
Great — then we can simplify the request store a bit.

==== - The worker will accumulate the full generated output for each request in memory until it reaches a terminal state (COMPLETED/FAILED/CANCELED) and the caller retrieves it. ====
* Polling status can still expose “tail”/metrics, but internally we keep the whole thing.

==== - This reduces complexity (no chunk files, no reassembly, no partial paging). ====
* It does mean you’ll want clear lifecycle cleanup: - once the caller calls get_result(id) (or an explicit release(id)), the worker can free stored output and metadata. - optionally a TTL GC for abandoned requests.

==== Even if “whole outputs” are fine, I’d add an optional per-worker max_output_chars (or None for unlimited). Not because you expect it normally, but because a runaway loop + high max_tokens could otherwise chew RAM. ====

Everything else stays the same: internal streaming feeds the accumulator, loop detector watches completed lines, and slot/backpressure is unchanged.