Editing Openai/691c1dba-9228-800f-8463-13b3a9006306 (section)

==== - Model choice: For your 2080 Supers, start with a quantized 7B (Llama 2, Mistral 7B variants) — they’re fast and cheap, high quality. Consider GPTQ quantized 13B only after you master quantization. ====
* GPU usage: Use 1 GPU for the model. Use the second GPU for embeddings (if supported) or other parallel processes (FAISS building, other small models).
* Persistence: ingots are saved to a local SQLite DB (yaiy_ingots.db). Back that up.
* Security: Keep models and ingot DB on encrypted disk, especially if you handle sensitive material.
* Extending: You can replace call_local_llm content with direct llama.cpp subprocess calls for better latency/no-HTTP.

If you want, I’ll now:
* (A) produce a ready-to-run llama.cpp subprocess version of hpl_runner.py (calls ./main binary directly), or
* (B) produce a text-generation-webui-specific payload adapter (the code above expects {"text":...} or JSON; I can tune it precisely to your webui version), or
* (C) walk you step-by-step over a remote shell session-style list of commands tuned to your OS.

Which next? Say A, B, or C, or ask for any tweak.