One Year After ChatGPT’s Launch, Has Generative AI Fulfilled its Promise?

When ChatGPT launched last November, it was hard to escape headlines and forecasts saying that generative AI (GenAI) would disrupt countless jobs and almost every industry. Where do we stand a year later? The reality is that, as with most new technologies, gauging its impact is more complicated than early prognosticators thought it would be. For example, the choruses predicting that Google search would suffer when Microsoft incorporated ChatGPT into Bing have largely fallen silent as Google’s market share held steady.¹ And, ChatGPT’s meteoric rise in traffic – it gained 100 million users in its first few months – actually reversed over the summer.² While traffic did pick up again after that, it hasn’t yet met the highs of this spring.

Generative AI still shows great promise, but its transformational power for large enterprises has so far been limited. Why? While there are numerous challenges, inaccuracy, cybersecurity, intellectual-property infringement, and regulatory compliance are the four most commonly cited risks that organizations are working to mitigate.³ The good news: solutions are coming that will likely make it easier for enterprises to move forward over the coming year.

Inaccuracy. While GenAI might be great at replying to natural language queries with quick human-like, conversational responses, those answers aren’t 100% reliable and trustworthy. Some startups are trying to remedy that. In August, AI startup Arthur launched Arthur Bench, an open-source tool for evaluating and comparing the performance of generative AI large language models (LLMs) on criterion like accuracy and how easily the text they generate can be understood.⁴ Launched in June, Galileo’s LLM Studio helps diagnose and fix issues with LLMs, by detecting hallucinations (i.e., when the LLM makes up facts) and incorrect predictions. The Studio is also working to address some other issues that trouble enterprise users. The platform allows data scientists to compare multiple prompts to find the one that will produce the most accurate and coherent response and can also estimate the costs of calls to external services like OpenAI.⁵ Many companies have also found that just by using retrieval-augmented generation (RAG) -- an AI framework for retrieving facts from an external knowledge base to ground LLMs on accurate, up-to-date information -- they are able to significantly reduce hallucinations and get more accurate responses.⁶

Cybersecurity. The early promise of ChatGPT was so great that executives from all kinds of companies rushed to figure out how to use it before realizing it was not an enterprise-ready product. A major concern was whether companies could trust this new technology with their proprietary data. That worry was compounded in April when OpenAI had a bug that leaked sensitive ChatGPT user data as well as chat history titles.⁷ Security issues like these are beginning to be addressed. This past summer, Microsoft and OpenAI released enterprise-friendly versions of ChatGPT designed to protect proprietary data by doing things like not using business data and conversations for retraining models.⁸ Also, companies can now choose from more open-source LLMs which obviate the need to share data with third parties.⁹

Intellectual-property infringement. GenAI introduces all kinds of IP issues. Should artists get paid when AI developers use their work to train models? Given that AI-generated work is not eligible for copyright, who owns the images or text created by an AI tool?¹⁰ To tackle these kinds of issues, YouTube is working with Universal Music Group to determine how artists should be compensated when their work is used by AI tools.¹¹ To help assuage some of the legal concerns that companies face when using GenAI, Microsoft, Adobe, Google, and OpenAI have each said that they will pay legal damages, with possible limitations, on behalf of customers using their GenAI products if the customers are sued for copyright infringement.¹²

Regulatory compliance. China was one of the first countries to regulate GenAI.¹³ New rules that took effect in August say that developers “bear responsibility” for the output created by their AI and that developers are liable if their training data infringes on someone else’s intellectual property. Chinese regulations also stipulate that AI services must be designed to generate only “true and accurate” content. The AI Act approved by the European Parliament in June requires GenAI developers to disclose if their models used copyrighted data for training and if content was produced using generative AI.¹⁴ After meeting with technology executives in September, Senate Majority Leader Chuck Schumer said that a workable AI bill needs to promote US innovation, ensure national and economic security, tackle copyright concerns and set transparency standards for AI companies.¹⁵ In October, The White House unveiled the first ever AI executive order on safe, secure, and trustworthy AI.¹⁶ It’s still early days for generative AI, and companies will need to stay on top of regulatory discussions and decisions as they work with this new technology.

There’s another challenge that has slowed the pace of GenAI adoption but that is harder for companies to do much about: hardware scarcity. Nvidia can’t meet demand for its much sought-after AI chips, and waitlists sometimes stretch to almost a year to access these chips at cloud computing companies like Google, Microsoft, and Amazon.¹⁷ In March 2023, Microsoft itself had to ration its internal teams’ access to AI hardware to ensure it had enough capacity to handle both Bing’s GPT-4 powered chatbot and its upcoming new Office tools.¹⁸ Manufacturing chips is an enormous undertaking -- building a semiconductor production facility today takes at least two years and $10 billion¹⁹ -- so this scarcity will likely last at least a few more years.

Given These Challenges, How Are Financial Services Firms Moving Forward with GenAI?

While the roadblocks to rapid GenAI adoption are a lot more evident a year after ChatGPT’s launch, the solutions are still being ironed out. That’s holding some financial firms back. But many firms are pushing ahead, with an understanding that despite the risks involved with LLMs today there are still many ways to take advantage of this technology. Their focus is on:

Internal use cases that keep humans in the loop. The “AI @ Morgan Stanley Assistant” went “fully live” in September. Using OpenAI’s technology, employees can now query the bank’s “intellectual capital,” a database of about 100,000 research reports and documents, and get back human-sounding responses to their questions.²⁰ In March, Citadel Securities said that it was negotiating an enterprise-wide license to use OpenAI’s ChatGPT tool for everything from helping developers write better code to translating software between languages.²¹ Financial services firms are also considering external use cases, though those may be further out. JP Morgan, for example, is using ChatGPT internally to analyze Federal Reserve speeches for potential trading opportunities but has also applied to trademark IndexGPT, an investment advice chatbot for customers.²²

Exploring the tradeoffs between open-source and commercial LLMs. Most organizations have decided that it’s too costly to train an LLM from scratch. Instead, they are fine-tuning existing models or using retrieval-augmented generation to augment LLM prompts with their own data, as mentioned above. The open question when it comes to using an existing model is whether to use a pre-trained open-source model like Meta’s LLaMA 2 or a commercially available one like GPT-4. Open-source models often provide greater flexibility and customization and allow companies to mitigate data security and privacy issues because they can be self-hosted. But commercial LLMs can expediate the launch of a GenAI application and allow companies to offload technology management, as they do with cloud computing (See Figure 1). How do we see this playing out? It’s likely that most large companies will pursue a hybrid strategy and pick open-source LLMs for projects using more confidential data or where more customization is needed and commercial models for projects where data security is less of a concern or to get something up and running quickly.

Questions to Consider

How will companies determine when to use open-source vs commercial LLMs? What is the best way to fine-tune LLMs with company data? How might the right strategy evolve over time or differ for certain types of projects? What strategies can companies implement to reduce costs when using commercial LLMs?

In getting started with this new technology, how should companies decide which GenAI projects to prioritize? Should companies prioritize internal or external use cases? Are there certain ways companies could safely and securely use this technology directly with their customers? How can we make sure people understand the limits of GenAI, such as its hallucinations?

What new roles and skills should companies look for as they explore experimenting with and implementing LLMs? Searching LinkedIn in October for “Generative AI” jobs in the US returned over 400 results. One of them was a job opening for a Generative AI QA Lead to provide best practices for generative AI testing and validation. What other jobs and skillsets may be needed to build products and processes most effectively with this new technology?