eBay introduced its own large language models (LLMs) tailored for the e-commerce sector in a paper published on June 17, 2024. The models, named LiLiuM 1B, 7B, and 13B, were developed in-house to meet eBay’s specific needs in translation and other e-commerce applications. These models aim to eliminate dependency on third-party LLMs within eBay and were trained on a dataset of 3 trillion tokens, including general and e-commerce-specific texts in multiple languages. eBay also developed their own tokenizer and model vocabulary customized for e-commerce tasks. The authors of the paper highlighted that their models perform as well as or better than popular LLaMA-2 models, particularly excelling in non-English machine translation and e-commerce-specific applications. They attribute this performance boost to the inclusion of non-English and e-commerce-specific data during pretraining. The authors plan to use these models as a foundation for fine-tuning and instruction-tuning to eliminate dependencies on external models. Future efforts will focus on enhancing the data pipeline, incorporating more eBay-specific data, training larger models, and exploring the Mixture-of-Experts architecture for improved efficiency.
Source link
Source link: https://slator.com/ebays-new-in-house-large-language-model-for-e-commerce-can-also-translate/
eBay launches new language model for e-commerce translation #eBayLanguageModel
