Editing Openai/693c0f4f-255c-8008-92e9-0cd44c6d6226 (section)

===== 1.3 Gap and opportunity =====

DnD learns a mapping:

: prompts from dataset → LoRA weights,
: supervised only in weight space.

Delta Activations gives:

: finetuned model (or LoRA) → behavioral embedding δ.

These two lines almost meet, but not quite:
* DnD knows how to go from data to weights, but doesn’t explicitly use any behavioral summary.
* Delta Activations knows how to go from weights to behavior, but doesn’t feed back into adapter generation.

Idea: use Delta Activations as auxiliary behavioral supervision for the data→LoRA generator, to make training easier and improve generalization.

You won’t replace weight-level supervision with δ (good instinct) – you’ll add δ as an extra, low-dimensional, behavior-aligned target.