Preeti
The OpenVINO toolkit has made significant improvements in response to user feedback, including expanding the ecosystem to cover additional scenarios and use cases. One key update is the introduction of Generative AI, particularly for AI assistants capable of generating text using Large Language Models (LLMs). A new package, openvino-genai, has been introduced to support LLMs, making it easier to build pipelines for various types of models. The toolkit now offers LLM-specific APIs that simplify the generation process, reducing the amount of code needed for implementation.
In terms of deployment, the OpenVINO Model Server (OVMS) now supports serving LLMs efficiently through continuous batching. Additional support has been added for serving models via TorchServe and Nvidia Triton Inference Server. Performance optimization efforts have focused on AI PCs, GPUs, and CPUs, with improvements in latency and memory footprint for various models.
The release also includes new notebooks and model support, such as mil-nce and openimages-v4-ssd-mobilenet-v2. The OpenVINO 2024.2 release is now available, with a roadmap for future features and enhancements. Intel technologies may require specific hardware, software, or service activation, and results may vary. Trademarks and brands mentioned are the property of their respective owners.
Source link
Source link: https://medium.com/openvino-toolkit/introducing-openvino-2024-2-ec7c0c857d00?source=rss——ai-5
GIPHY App Key not set. Please check settings