Fast and cost-effective LLaMA 2 fine-tuning with AWS Trainium
AWS Machine Learning
OCTOBER 5, 2023
Our high-level training procedure is as follows: for our training environment, we use a multi-instance cluster managed by the SLURM system for distributed training and scheduling under the NeMo framework. For detailed guidance of downloading models and the argument of the preprocessing script, refer to Download LlamaV2 dataset and tokenizer.
Let's personalize your content