Preeti
A study conducted by the Federal Reserve Research in Kansas City and Richmond evaluates the performance of local large language models (LLMs) in analyzing financial texts compared to closed-source, cloud-based models. The study consists of two main exercises. The first exercise benchmarks local LLM performance in analyzing financial and economic texts, introducing new benchmarking tasks and exploring refinements needed to improve performance. Results suggest that local LLMs are effective for general NLP analysis of financial and economic texts.
In the second exercise, local LLMs are used to analyze the tone and substance of bank earnings calls in the post-pandemic era, including calls during the banking stress of early 2023. The analysis includes topics discussed, overall sentiment, temporal orientation, and vagueness in remarks from bank earnings calls. Following the banking stress of early 2023, bank calls tended to focus on similar topics and conveyed a less positive sentiment.
The study provides insights into the effectiveness of local LLMs in analyzing financial texts and bank earnings calls, highlighting their potential for use in NLP analysis. The paper can be accessed for further details on the study.
Source link
Source link: https://www.frbsf.org/research-and-insights/publications/system-research-kansas-city-fed/2024/06/evaluating-local-language-models-an-application-to-financial-earnings-calls
GIPHY App Key not set. Please check settings