← Quit0 / 0 correct
Roughly how much GPU memory does full fine-tuning of a 7B model with Adam in mixed precision need for weights, gradients and optimizer state alone?
Roughly how much GPU memory does full fine-tuning of a 7B model with Adam in mixed precision need for weights, gradients and optimizer state alone?