← Quit0 / 0 correct
How does LoRA make fine-tuning cheap?
x ─┬─▶ W (frozen, d×k) ──┐
│ (+)─▶ y
└─▶ A (d×r) ▶ B (r×k)─┘
r ≪ d, k (e.g. 8)How does LoRA make fine-tuning cheap?
x ─┬─▶ W (frozen, d×k) ──┐
│ (+)─▶ y
└─▶ A (d×r) ▶ B (r×k)─┘
r ≪ d, k (e.g. 8)