Preeti
The blog post discusses the development of LangProp, a code optimization framework utilizing Large Language Models (LLMs) applied to driving, presented at the LLM Agents workshop at ICLR in May 2024. The challenge of applying LLMs to driving lies in their size and speed, as well as the need for detailed observations and spatial control actions. LangProp aims to use LLMs to generate and improve driving code without fine-tuning the neural network itself.
The LangProp framework is designed similar to PyTorch Lightning, where the LLM acts as an optimizer guiding code parameters to reduce loss. By automating software optimization in a data-driven way, LangProp unifies Symbolic AI with Machine Learning approaches, offering a new method to learn symbolic systems and automate their improvement.
The blog post details the training process of LangProp in a driving simulation environment using CARLA, where the LLM generates code for driving actions based on input state information. Through a combination of imitation learning and reinforcement learning techniques like DAgger, LangProp iteratively improves driving policies, surpassing expert agents in performance.
The post concludes by highlighting the potential applications of LangProp in various software optimization tasks beyond autonomous driving, inviting further exploration and development in this space. The paper on LangProp presented at ICLR 2024 is referenced for more in-depth information on the framework and its implementation.
Source link
Source link: https://towardsdatascience.com/making-llms-write-better-and-better-code-for-self-driving-using-langprop-99c6c3dc9508
GIPHY App Key not set. Please check settings