Editing Openai/693c0f4f-255c-8008-92e9-0cd44c6d6226 (section)

==== : ====
* If you literally mean only δ, no dataset at inference: - You can generate some LoRA, but you don’t know what it solves. - It’s good for exploring / composing existing LoRAs, not for new tasks. - It does not give the “no-training adaptation to arbitrary new datasets” that DnD aims for.
* If you’re willing to: - either (a) use δ for existing LoRAs, and - (b) maybe combine δ with dataset prompts for new tasks, then it becomes a cool meta-learning & composition tool: - δ as a low-dim task/behavior code, - a generator G(δ) as a LoRA decoder, - possible compositional tricks (δ_add, δ_mix) in that space.

So conceptually it’s promising, but its main weakness is:
δ depends on having some model that already does the task, so by itself it can’t replace dataset-conditioned LoRA generation if your goal is “not training but generating LoRA.” It’s a better fit for “not training a new LoRA but re-mixing/compressing existing LoRAs.”

If you want, next step we could sketch a toy experimental design for:
* build δ-space over a bunch of LoRAs,
* train G(δ) → LoRA,
* test: does δ_math + δ_code → LoRA that actually improves on math+code benchmarks?