People's predictions are no match for ChatGPT's predictions? Research shows: ChatGPT reliably predicts stock prices based on news headlines

Last December, the artificial intelligence chatbot program developed by OpenAI, ChatGPT, reached one million user registrations within five days of its launch, sparking a global wave of interest in AI technology. This article will explore an experiment conducted by Alejandro Lopez-Lira, a professor at the University of Florida, using ChatGPT to analyze financial news headlines and determine whether the news is positive or negative for stock prices, providing a glimpse into the limitless potential of Large Language Models (LLMs) in predicting finance and stocks.

The Soaring Popularity of ChatGPT

ChatGPT, short for Chat Generative Pre-trained Transformer, is an artificial intelligence chatbot program developed by OpenAI. It is based on the architecture of GPT-3.5 and GPT-4, using the large language model (LLM) for reinforcement learning training. ChatGPT can interact with humans using natural language and handle complex tasks, providing detailed and clear answers in multiple professional fields. It can even pass exams in various professional fields such as law and medicine in different countries.

The text generated by ChatGPT closely resembles human writing, sparking discussions and anxiety about whether "AI will replace humans." As ChatGPT seems capable of handling white-collar jobs that were previously considered difficult to automate, it has further fueled these discussions and concerns.

Regarding user growth, it became the fastest program to reach a million users in history—achieving this milestone in just two months.

Surprising Performance of ChatGPT in Stock Price Prediction?

Professor Alejandro Lopez-Lira from the University of Florida used ChatGPT in an experiment to analyze financial news headlines and let ChatGPT determine whether these news items would have a positive or negative impact on stock prices. In a recent paper, it was noted that ChatGPT's ability to predict the direction of returns for the next trading day outperformed random predictions by a significant margin.

This experiment utilized state-of-the-art artificial intelligence technology with more powerful computers and superior datasets. Such AI models may exhibit "Emergent Ability," which refers to unforeseen capabilities that emerge during model building.

Note: Emergent Ability is a phenomenon in large language models (LLMs) where the model develops abilities that were not anticipated during its construction, such as basic social knowledge and reasoning skills. The update to GPT-4 has brought significant attention to this phenomenon, but there is currently no clear answer in the academic community as to why AI exhibits these emergent abilities.

How Does ChatGPT's Financial Prediction Experiment Work?

In this experiment, Lopez-Lira and his partner, Yuehua Tang, retrieved over 50,000 financial news headlines related to companies listed on the New York Stock Exchange, Nasdaq, and small exchanges from a data provider.

The experiment began in October 2022, which also served as the cutoff date for ChatGPT's training data—meaning ChatGPT had not seen this data during its original training. They then input these headlines along with prompts into ChatGPT 3.5.

Outstanding Performance of ChatGPT

They then observed the performance of these stocks on the following day.

Lopez-Lira found that in almost all cases, ChatGPT's performance in predicting based on news headlines surpassed random predictions. Specifically, he found that ChatGPT's success rate in predicting the direction of stock price movement the next day was less than 1% with random predictions, but it was higher after reading relevant news headlines.

ChatGPT even outperformed commercial datasets with human sentiment scores. An example from the paper shows: a headline about a company settling a lawsuit and paying a fine, which had a negative sentiment, but ChatGPT correctly deduced that this was actually good news.

Lopez-Lira mentioned that some hedge funds have contacted him to inquire about his research. As institutions begin to integrate this technology, the predictive ability of ChatGPT in forecasting stock trends may decrease in the coming months, which is not surprising.

"As more people use these tools, the market will become more efficient, and the predictability of returns will decrease. So, I guess if I were to redo this experiment in five years, by the fifth year, the predictability of returns would be zero." Lopez-Lira stated.

Impact of ChatGPT on Financial Professionals?

If ChatGPT's emergent abilities can understand financial news headlines and their potential impact on stock prices, it could pose a risk to high-paying jobs in the financial industry. Goldman Sachs estimated in a report on March 26th that about 35% of financial jobs face the risk of automation by artificial intelligence.

"ChatGPT can understand human information, which almost guarantees that if the market is not fully reactive, there will be predictability in returns." Lopez-Lira expressed.

ChatGPT Still Has a Long Way to Go Before Replacing Humans

However, the results of this experiment also indicate that large language models (LLMs) still have a long way to go in completing many financial tasks. For example, this experiment did not involve target prices or require the model to perform any mathematical calculations. ChatGPT has often been criticized for fabricating numbers and calculations.

Text sentiment analysis has been widely regarded as a trading strategy in the presence of its specific dataset.

Note: Text sentiment analysis refers to the use of methods such as natural language processing, text mining, and computational linguistics to identify and extract subjective information from the original material.

Lopez-Lira mentioned that he was surprised by the results of the experiment, adding that it indicates advanced investors have not yet incorporated machine learning technologies like ChatGPT into their trading strategies.

"From a regulatory perspective, if we let machines read news headlines, the importance of news headlines will increase. And it's worth noting, should everyone have the permission to use machines like GPT? Secondly, this will definitely have some impact on financial analysts' employment. The question arises, do I want to spend money to hire an analyst, or do I just input text messages into the model?" Lopez-Lira remarked.

Will ChatGPT Bring More Freedom and Happiness to Humans?

In less than half a year since its launch, ChatGPT has demonstrated remarkable abilities in various fields, even exhibiting "Emergent Ability" and other phenomena that humans cannot yet comprehend. The results of this experiment also raise expectations for how much impact ChatGPT will have in the financial sector in the future. Nevertheless, it still has many weaknesses: it fabricates numbers and calculations and can even generate misinformation, indicating that this new technology, whether good or bad, will deeply influence the society we live in.

As Professor Lopez-Lira mentioned: "Should everyone have the permission to use machines like GPT?" Whether such technological innovations will make it easier for those who currently have priority access to accumulate assets and widen the wealth gap remains to be seen. Should the application of AI technology prioritize humans and aim to provide a higher quality of life for humanity? Issues related to regulation, ethics, and social justice that arise from this discussion are perhaps aspects we should focus on and discuss more intensively.

Does ChatGPT Possess Intelligence?

Regarding the hot topic on social media—whether ChatGPT already possesses intelligence?

Is it merely a machine that generates text without "intelligence," or can it assist humans in solving complex problems, thereby addressing societal issues and enhancing human well-being?

Faced with such questions, Microsoft researcher Sebastien Bubeck did not provide an answer but hoped that we could rethink "what intelligence is" and leave room for readers to define it for themselves.

Sebastien Bubeck stated in his talk on "Sparks of AGI: early experiments with GPT-4":

"As a society, as humans, what should we conclude from this? We must move beyond discussions at the level of 'Is this copying and pasting or statistical data?' We need to move beyond these discussions. The train has left the station. If we are constantly stuck on these questions, we will miss the truly important questions."