Preeti
Google’s latest artificial intelligence models, Gemini 1.5 Pro and Gemini 1.5 Flash, are designed to process large amounts of context data, but recent studies have raised doubts about their effectiveness. While these models can technically handle data in large context windows, they may struggle to truly understand it. One study found that Gemini, along with other vision language models, loses performance as the visual context length grows. Another study revealed that no open-weight model performs above random chance.
The limitations of Gemini were highlighted in a test called NoCha, where Gemini 1.5 Pro scored 48.1% and Gemini 1.5 Flash scored 34.2%. These scores were lower than the best OpenAI and Claude models, and significantly lower than the 97% scored by humans on the same test. While Gemini may still be useful for specific tasks like finding information in a sentence, it falls short when it comes to understanding and processing data on a larger scale.
Despite Google’s claims about the capabilities of Gemini, these studies show that artificial intelligence still has a long way to go before it can match human reasoning. While AI models like Gemini may be faster at processing data, they are not always more accurate. The findings suggest that AI models like Gemini have limitations in their ability to truly understand and analyze complex information.
Source link
Source link: https://www.androidcentral.com/apps-software/is-geminis-expanded-context-window-as-useful-as-we-thought
GIPHY App Key not set. Please check settings