Editing Openai/693c0f4f-255c-8008-92e9-0cd44c6d6226 (section)

==== 2. Goals and Research Questions ====

Goal: Improve data-to-LoRA generation by incorporating Delta Activations as an auxiliary task, making the generator more behavior-aware and easier to train.

RQ1 – In-domain reconstruction:
Does delta-based auxiliary supervision improve how well a prompt-conditioned generator reproduces known LoRAs and their behavior on the same tasks?

RQ2 – Zero-shot cross-task generalization:
Given only prompts from a new dataset, does the delta-augmented generator produce LoRAs that perform better (or at least more robustly) than a DnD-style baseline?

RQ3 – Robustness and sample efficiency:
Does δ supervision help when training data (number of prompt–checkpoint pairs) is small, or when LoRA targets are noisy?

RQ4 – Compositional task behavior:
If we interpolate or add deltas (e.g., δ_math + δ_code), and decode them into LoRAs, do we get meaningful joint capabilities?