Snowflake’s new LLM outperforms Meta’s Llama 3 in training. #AI

Snowflake, a data warehousing cloud vendor, has announced an open-source AI model called Arctic that is claimed to be more efficient than Meta’s Llama 3 at enterprise tasks like SQL coding for database retrieval. Arctic, a large language model, is said to perform on par or better than Llama 3 and Llama 2 while using less training compute budget. Snowflake has released Arctic’s parameters and model code under the Apache 2.0 open-source license, along with training data and research insights. The Arctic program is available immediately as an inference service on Snowflake’s Cortex offering and will be available on other platforms like Amazon AWS and Hugging Face. Snowflake’s CEO sees this as a significant moment for the company’s AI research team. The approach taken by Snowflake’s AI leads combines a transformer attention-based model with a “mixture of experts” (MoE) to conserve computing and memory needs. Snowflake’s variant is called a “Dense – MoE Hybrid Transformer.” While Snowflake has not published a formal paper, technical details are available on GitHub. The company has also published blog posts and a demo of Arctic providing inference on the Hugging Face repository. Arctic follows Snowflake’s recent release of a family of text-embedding models, which excel in word grouping for search purposes.

Source link

Source link: https://www.zdnet.com/article/snowflake-says-its-new-llm-outperforms-metas-llama-3-on-half-the-training/