Title: Manual curation vs. AI distillation: Lessons learned for instruction following and feedback fine-tuning Abstract: There has been a slew of work in training helpful conversational agents using Large language models (LLMs). These models draw upon diverse datasets, including open-source repositories, private data, and even synthetic data generated from LLMs. However, curating datasets for SFT and RLHF involves critical decisions, such as defining task distributions, data volume, prompt len…