Editing Openai/693c0f4f-255c-8008-92e9-0cd44c6d6226 (section)

===== Workflow: =====
# Precompute vfv_fvf (Δ) for a library of finetuned LoRAs.
# For a new task: - Do some training or few-shot finetune to get a temporary model, - Compute its Δ embedding vtaskv_{\text{task}}vtask, - Use nearest neighbors in Δ-space to select existing LoRAs, maybe merge them.

This is actually very close to what the Delta paper already explores (few-shot task embeddings + model selection / merging). 2509.04442v1

Pros:
* You only ever train small finetunes (few-shot) to get a task embedding.
* The heavy adaptation is done by reusing LoRAs you already have.

Cons:
* You still need real LoRAs in your library – δ alone does not adapt the base model.
* If your library doesn’t already contain something close to the new task, retrieval can’t magically create it.

Compared to DnD “generate a fresh LoRA for any new dataset”, this is strictly weaker in expressiveness; it can’t cover capabilities that aren’t already somewhere in the library.