Utilizing Large Language Models for Advanced Analysis and Prediction #NLP

In this article, we investigate the potential of Large Language Models (LLMs) for numerical data analysis, aiming to address the following key questions:

1. What types of data sources and analyses can LLMs handle?
2. How can companies leverage LLMs to overcome bottlenecks in analysis and reporting?
3. What implications does this have for the career evolution of data scientists?
4. What enablers and improvements are necessary for the wider adoption of LLMs in numerical data analysis?

While LLMs are primarily language models trained on text, recent models like GPT-4o have demonstrated the ability to apply code generation and built-in compute environments to address numerical data analysis challenges. We explore the capabilities of LLMs in this domain and discuss the implications for businesses and data professionals.

Data:

• We use a sample dataset from IBM, which can be found on Kaggle link.
• The dataset contains various factors that influence employee attrition. Currently, only flat files are supported.

Tool:

• We use GPT-4o, which is accessible through OpenAI’s ChatGPT subscription.

Descriptive Analysis:

This topic aims to gain insights into past events and the influence of critical factors. To achieve this, I posed questions to comprehend the characteristics of employees who left the company and contrasted them with those who stayed. The LLM demonstrated its capability by distinguishing between numerical and categorical data types, delivering accurate results accompanied by insightful interpretations and visual representations.

Diagnostic Analysis:

The primary goal of diagnostic analysis is to gain a deeper understanding of the reasons behind employee departures. I was particularly impressed by the integration of basic statistical methods and the LLM’s ability to draw on its reasoning capabilities to generate insights and potential interventions.

In my quest for deeper insights, I challenged the LLM to provide a list of crucial factors contributing to employee attrition, supported by robust statistical analysis. Impressively, the LLM not only outlined the necessary steps but also elaborated on the processing methods involved. The results it presented were remarkably accurate and insightful, further solidifying the LLM’s capabilities.

Predictive analysis delves into the realm of machine learning, where the goal is to construct a model capable of forecasting the probability of specific events. In this case, the focus is on employee attrition. The LLM demonstrated remarkable clarity in outlining the necessary steps for model construction, encompassing algorithm selection, performance metric evaluations, and recommendations for further enhancements.

In response to the request for implementing model improvement measures, the LLM successfully delivered a range of enhancements. These included an upgraded logistic regression model and a random forest model, along with their respective performance metrics. Unfortunately, due to limited computing resources, the LLM was unable to implement some of the more advanced measures such as feature engineering and hyperparameter tuning.

It’s evident that the LLM environment has encountered computational limitations. However, it was able to provide the output and model code for local implementation, likely with the assistance of a human data scientist. I refrained from conducting prescriptive analysis or recommending interventions since the LLM had already offered insights to reduce attrition. In the future, I anticipate full integration with payroll systems to enable automatic adjustments to benefits and pay raises.

It’s noteworthy that when I attempted a similar exercise with Google’s Gemini AI, while it could partially read the dataset, it promptly declared that it’s designed solely for textual data processing. Therefore, while GPT-4 has demonstrated success, there’s still room for significant improvement, which we can hopefully expect in the near future.

Summary of Findings:

1. LLMs’ Data Handling Capabilities:

• LLMs can process small datasets in flat files and perform various data science tasks, excluding model tuning.
• Future advancements may enable LLMs to connect to extensive databases and facilitate a complete spectrum of data science activities, encompassing decision-making and action execution through systems like CRM and HR.

2. Overcoming Data Science Bottlenecks with LLMs:

• LLMs have the potential to address up to 80% of simple, repetitive data science requests, significantly reducing turnaround times due to the scarcity of skilled data scientists.

3. Impact on Data Scientists’ Careers:

• Data scientists must specialize in domains that cannot be automated, such as model explainability, fine-tuning, and engineering, to maintain relevance amid the adoption of LLMs.

4. Enablers and Enhancements for Broader LLM Adoption:

• Trust in LLMs is essential for their widespread adoption in automating data science activities. This trust can be established through explainability and appropriate governance measures.
• Security is a prime consideration, and the introduction of localized LLMs in private clouds without external communication could pave the way for broader adoption.

Upcoming Article:

• My next article will delve deeper into the enablers and improvements required for the broader adoption of LLMs in numerical data analysis.