Google has introduced new generative AI models called Gemini 1.5 Pro and 1.5 Flash, which are capable of processing and analyzing vast amounts of data with high accuracy. These models can summarize lengthy documents and complex data sets, making them groundbreaking tools in the field of AI. However, recent research has raised doubts about the effectiveness of these models, particularly in processing long-context content.
Despite Google’s claims about Gemini’s ability to handle large datasets, recent studies have shown that the models struggle to understand and process extensive data inputs. In tests, Gemini 1.5 Pro and 1.5 Flash were found to correctly answer questions about large datasets only 40-50% of the time. This raises concerns about the models’ actual understanding of the content they are processing.
The context window of a model refers to the amount of input data it can consider before generating a response. The latest versions of Gemini can handle up to 2 million tokens of context, making them the largest commercially available AI models in terms of context capacity. However, demonstrations of Gemini’s capabilities may have exaggerated the models’ real-world abilities.
In one study, researchers tested the models’ ability to evaluate true/false statements about contemporary fiction books, which required comprehension of the entire plot. The models struggled to verify the accuracy of detailed statements about the books, indicating limitations in their understanding of complex content. Overall, while Gemini models show promise in data processing, further research is needed to fully understand their capabilities and limitations.
Source link
Source link: https://www.indiatvnews.com/amp/technology/news/google-s-gemini-models-are-they-as-good-as-claimed-2024-06-30-939600