in

#Training small language models for improved reasoning skills. #NLP

Teaching Small Language Models to Reason | by Cobus Greyling | Jul, 2024

Chain-Of-Thought Prompting has been successful in improving the reasoning capabilities of large language models, leading to the emergence of the Chain-Of-X phenomenon. Google Research explored how to create a CoT data ontology for existing datasets using LLMs and fine-tuned smaller Language Models on the CoT. They transferred reasoning capabilities from larger models to smaller ones through knowledge distillation, improving task performance in arithmetic, common sense, and symbolic reasoning datasets.

CoT prompting teaches Language Models to break down reasoning tasks into intermediate steps, significantly increasing task accuracy for large language models across various datasets. However, smaller LMs do not benefit from CoT prompting, often producing illogical results. This is attributed to certain abilities like semantic understanding and symbolic mapping only emerging in larger models.

Google Research proposed a two-step pipeline for CoT knowledge distillation, involving annotation with CoT reasoning using teacher models like PaLM 540B or GPT-3 175B, and fine-tuning student models with teacher forcing. This method eliminates the need for prompting during fine-tuning.

Overall, CoT prompting has shown promise in enhancing the reasoning capabilities of large language models, but its effectiveness diminishes in smaller models. The research highlights the importance of scale in developing advanced reasoning abilities in language models.

Source link

Source link: https://medium.com/@cobusgreyling/teaching-small-language-models-to-reason-9107d9881075

What do you think?

Leave a Reply

GIPHY App Key not set. Please check settings

OpenAI Launching the Ultimate AI Health Coach, Backed by BILLIONAIRES! (Longer Life)

Ultimate AI Health Coach by OpenAI, Supported by Billionaires! #LongerLife

Samsung Expands Next-Generation Care with AT&T and Introduces New AI-Powered Tools

Samsung partners with AT&T to enhance customer care services. #AIpowered