A highly capable language model called phi-3-mini has been developed, trained on 3.3 trillion tokens and containing 3.8 billion parameters. Despite its small size, it performs well on academic benchmarks and internal testing, comparable to larger models like Mixtral 8x7B and GPT-3.5. The model’s innovation lies in its dataset, which includes filtered web data and synthetic data, making it robust, safe, and suitable for chat formats.

The report discusses the progress of AI in scaling up models and datasets, with large language models now reaching trillion parameters. The phi-3-mini model, with its small size, can be deployed on a phone and achieve high performance. The research team behind phi-3-mini has also developed larger models, phi-3-small and phi-3-medium, which outperform phi-3-mini.

The model’s ability to run on a phone, specifically an iPhone with an A16 Bionic chip, is highlighted, with the quantized version of phi-3-mini occupying minimal memory and achieving high token generation rates. The researchers involved in this project are listed, and the paper is available for further reading.

Overall, the development of phi-3-mini represents a significant advancement in AI technology, with the potential to unlock a new era of computing. The research has been covered by Wired in an article titled “Pocket-Sized AI Models Could Unlock a New Era of Computing,” highlighting the importance and implications of this work in the field of artificial intelligence.

Source link

Source link: https://astrobiology.com/2024/07/tricorder-tech-a-highly-capable-language-model-locally-on-your-phone.html