The post NVIDIA NeMo-Aligner Enhances Supervised Fine-Tuning with Data-Efficient Knowledge Distillation appeared on BitcoinEthereumNews.com.

Peter Zhang Dec 18, 2024 09:40 NVIDIA NeMo-Aligner introduces a data-efficient approach to knowledge distillation for supervised fine-tuning, enhancing performance and efficiency in neural models. NVIDIA’s NeMo-Aligner has unveiled a new methodology for enhancing supervised fine-tuning (SFT) through data-efficient knowledge distillation. This innovative approach allows for the transfer of knowledge from a larger teacher model to a more compact student model, achieving comparable accuracy with reduced data requirements, according to NVIDIA. Advancements in Knowledge Distillation Knowledge distillation is a technique that has been widely used in pretraining scenarios but is less explored in the context of supervised fine-tuning. NeMo-Aligner aims to bridge this gap by leveraging knowledge distillation during SFT to enhance model accuracy and efficiency. The method achieves higher accuracy than standard SFT by utilizing only 70% of the training steps, as demonstrated in their experiments. Implementation and Benefits The NeMo-Aligner uses a KD-logit approach, where the student model is trained to match the teacher’s output logits. This technique, known as “dark knowledge,” provides a more informative gradient signal by understanding the similarities and dissimilarities across classes. The process involves preprocessing where the teacher model’s predictions are cached, and the student model is trained to align with these predictions, resulting in memory savings and faster training times. The approach significantly reduces the need for simultaneous loading of both teacher and student models, thus saving GPU memory. Instead, only the top-K logits of the teacher are stored, optimizing memory usage while maintaining detailed information transfer. Empirical Results Experiments conducted with the Nemotron-4 15B student model and a fine-tuned Nemotron-4 340B teacher model reveal that the KD-finetuned models outperform the vanilla SFT models in multiple benchmarks, including HumanEval, MBPP, and MATH. Notably, the KD-finetuned model requires fewer training tokens while achieving superior performance across six of seven evaluation metrics. The KD…

---

Filed under: News - @ December 18, 2024 10:22 am