June 12
Section Text 1.1
Artificial Intelligence

One Year After ChatGPT’s Launch, Has Generative AI Fulfilled its Promise?

By: Sarah HoffmaN | November 29, 2023
founderiepitching Bits & Blocks Clubblockchain301

It’s been a year since the launch of ChatGPT. For large enterprises, the transformational power of generative AI technology has so far been limited. Yet changes are coming to help alleviate some of the current challenges of this technology, and large firms are finding their path forward.


Wednesday, December 13, 2023

9:00 a.m. – 10:00 a.m. ET



Meeting ID: 994 3158 6099
Passcode: 253444

  • Facebook.
  • Twitter.
  • LinkedIn.
  • Print

When ChatGPT launched last November, it was hard to escape headlines and forecasts saying that generative AI (GenAI) would disrupt countless jobs and almost every industry. Where do we stand a year later? The reality is that, as with most new technologies, gauging its impact is more complicated than early prognosticators thought it would be. For example, the choruses predicting that Google search would suffer when Microsoft incorporated ChatGPT into Bing have largely fallen silent as Google’s market share held steady.1 And, ChatGPT’s meteoric rise in traffic – it gained 100 million users in its first few months – actually reversed over the summer.2 While traffic did pick up again after that, it hasn’t yet met the highs of this spring.

Generative AI still shows great promise, but its transformational power for large enterprises has so far been limited. Why? While there are numerous challenges, inaccuracy, cybersecurity, intellectual-property infringement, and regulatory compliance are the four most commonly cited risks that organizations are working to mitigate.3 The good news: solutions are coming that will likely make it easier for enterprises to move forward over the coming year.

Inaccuracy. While GenAI might be great at replying to natural language queries with quick human-like, conversational responses, those answers aren’t 100% reliable and trustworthy. Some startups are trying to remedy that. In August, AI startup Arthur launched Arthur Bench, an open-source tool for evaluating and comparing the performance of generative AI large language models (LLMs) on criterion like accuracy and how easily the text they generate can be understood.4 Launched in June, Galileo’s LLM Studio helps diagnose and fix issues with LLMs, by detecting hallucinations (i.e., when the LLM makes up facts) and incorrect predictions. The Studio is also working to address some other issues that trouble enterprise users. The platform allows data scientists to compare multiple prompts to find the one that will produce the most accurate and coherent response and can also estimate the costs of calls to external services like OpenAI.5 Many companies have also found that just by using retrieval-augmented generation (RAG) -- an AI framework for retrieving facts from an external knowledge base to ground LLMs on accurate, up-to-date information -- they are able to significantly reduce hallucinations and get more accurate responses.6

Cybersecurity. The early promise of ChatGPT was so great that executives from all kinds of companies rushed to figure out how to use it before realizing it was not an enterprise-ready product. A major concern was whether companies could trust this new technology with their proprietary data. That worry was compounded in April when OpenAI had a bug that leaked sensitive ChatGPT user data as well as chat history titles.7 Security issues like these are beginning to be addressed. This past summer, Microsoft and OpenAI released enterprise-friendly versions of ChatGPT designed to protect proprietary data by doing things like not using business data and conversations for retraining models.8 Also, companies can now choose from more open-source LLMs which obviate the need to share data with third parties.9

Intellectual-property infringement. GenAI introduces all kinds of IP issues. Should artists get paid when AI developers use their work to train models? Given that AI-generated work is not eligible for copyright, who owns the images or text created by an AI tool?10 To tackle these kinds of issues, YouTube is working with Universal Music Group to determine how artists should be compensated when their work is used by AI tools.11 To help assuage some of the legal concerns that companies face when using GenAI, Microsoft, Adobe, Google, and OpenAI have each said that they will pay legal damages, with possible limitations, on behalf of customers using their GenAI products if the customers are sued for copyright infringement.12

Regulatory compliance. China was one of the first countries to regulate GenAI.13 New rules that took effect in August say that developers “bear responsibility” for the output created by their AI and that developers are liable if their training data infringes on someone else’s intellectual property. Chinese regulations also stipulate that AI services must be designed to generate only “true and accurate” content. The AI Act approved by the European Parliament in June requires GenAI developers to disclose if their models used copyrighted data for training and if content was produced using generative AI.14 After meeting with technology executives in September, Senate Majority Leader Chuck Schumer said that a workable AI bill needs to promote US innovation, ensure national and economic security, tackle copyright concerns and set transparency standards for AI companies.15 In October, The White House unveiled the first ever AI executive order on safe, secure, and trustworthy AI.16 It’s still early days for generative AI, and companies will need to stay on top of regulatory discussions and decisions as they work with this new technology.

There’s another challenge that has slowed the pace of GenAI adoption but that is harder for companies to do much about: hardware scarcity. Nvidia can’t meet demand for its much sought-after AI chips, and waitlists sometimes stretch to almost a year to access these chips at cloud computing companies like Google, Microsoft, and Amazon.17 In March 2023, Microsoft itself had to ration its internal teams’ access to AI hardware to ensure it had enough capacity to handle both Bing’s GPT-4 powered chatbot and its upcoming new Office tools.18 Manufacturing chips is an enormous undertaking -- building a semiconductor production facility today takes at least two years and $10 billion19 -- so this scarcity will likely last at least a few more years.

Given These Challenges, How Are Financial Services Firms Moving Forward with GenAI?

While the roadblocks to rapid GenAI adoption are a lot more evident a year after ChatGPT’s launch, the solutions are still being ironed out. That’s holding some financial firms back. But many firms are pushing ahead, with an understanding that despite the risks involved with LLMs today there are still many ways to take advantage of this technology. Their focus is on:

Internal use cases that keep humans in the loop. The “AI @ Morgan Stanley Assistant” went “fully live” in September. Using OpenAI’s technology, employees can now query the bank’s “intellectual capital,” a database of about 100,000 research reports and documents, and get back human-sounding responses to their questions.20 In March, Citadel Securities said that it was negotiating an enterprise-wide license to use OpenAI’s ChatGPT tool for everything from helping developers write better code to translating software between languages.21 Financial services firms are also considering external use cases, though those may be further out. JP Morgan, for example, is using ChatGPT internally to analyze Federal Reserve speeches for potential trading opportunities but has also applied to trademark IndexGPT, an investment advice chatbot for customers.22

Exploring the tradeoffs between open-source and commercial LLMs. Most organizations have decided that it’s too costly to train an LLM from scratch. Instead, they are fine-tuning existing models or using retrieval-augmented generation to augment LLM prompts with their own data, as mentioned above. The open question when it comes to using an existing model is whether to use a pre-trained open-source model like Meta’s LLaMA 2 or a commercially available one like GPT-4. Open-source models often provide greater flexibility and customization and allow companies to mitigate data security and privacy issues because they can be self-hosted. But commercial LLMs can expediate the launch of a GenAI application and allow companies to offload technology management, as they do with cloud computing (See Figure 1). How do we see this playing out? It’s likely that most large companies will pursue a hybrid strategy and pick open-source LLMs for projects using more confidential data or where more customization is needed and commercial models for projects where data security is less of a concern or to get something up and running quickly.

Advantages to Using an Open-source LLM Advantages to Purchasing a Commercial LLM
  • Customization
  • Intellectual property
  • Data privacy and security
  • Potential cost savings long-term
  • More control over latency of the LLM
  • No vendor “lock-in”
  • Faster time-to-market for GenAI applications
  • Requires fewer experts and technology resources
  • Cost savings (short term)
Figure 1: Some tradeoffs between open-source and commercial LLM technology | Source: FCAT Research

Questions to Consider

How will companies determine when to use open-source vs commercial LLMs? What is the best way to fine-tune LLMs with company data? How might the right strategy evolve over time or differ for certain types of projects? What strategies can companies implement to reduce costs when using commercial LLMs?

In getting started with this new technology, how should companies decide which GenAI projects to prioritize? Should companies prioritize internal or external use cases? Are there certain ways companies could safely and securely use this technology directly with their customers? How can we make sure people understand the limits of GenAI, such as its hallucinations?

What new roles and skills should companies look for as they explore experimenting with and implementing LLMs? Searching LinkedIn in October for “Generative AI” jobs in the US returned over 400 results. One of them was a job opening for a Generative AI QA Lead to provide best practices for generative AI testing and validation. What other jobs and skillsets may be needed to build products and processes most effectively with this new technology?

  • Facebook.
  • Twitter.
  • LinkedIn.
  • Print
The opinions provided are those of the author and not necessarily those of Fidelity Investments or its affiliates. Fidelity does not assume any duty to update any of the information. Fidelity and any other third parties mentioned are independent entities and not affiliated. Mentioning them does not suggest a recommendation or endorsement by Fidelity.

The information regarding ChatGPT and other AI tools is for informational purposes only and is not intended to constitute a recommendation, development, security assessment advice of any kind. Consider your own use case carefully and understand the risks before utilizing a generative AI tool.
1 https://www.wsj.com/tech/ai/microsoft-bing-search-artificial-intelligence-google-competition-6e51ec04
2 Ruby, Daniel. “ChatGPT Statistics for 2023: Comprehensive Facts and Data.” Demandsage, 28 Apr. 2023.
3 https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in-2023-generative-ais-breakout-year
4 https://venturebeat.com/ai/arthur-unveils-bench-an-open-source-ai-model-evaluator/
5 https://venturebeat.com/automation/galileo-launches-llm-studio-to-revolutionize-ai-adoption-in-enterprises/
6 https://thenewstack.io/reduce-ai-hallucinations-with-retrieval-augmented-generation/
7 https://www.engadget.com/openai-says-a-bug-leaked-sensitive-chatgpt-user-data-165439848.html
8 https://www.wsj.com/tech/ai/openai-launches-business-version-of-chatgpt-that-competes-with-microsoft-6ea3ff2f
9 Meta. “Llama 2.” Meta AI,
10 Mattei, Shanti Escalante-De. “US Copyright Office: AI Generated Works Are Not Eligible for Copyright.” ARTnews.com, 21 Mar. 2023.
11 https://www.wsj.com/tech/how-frank-sinatra-and-yo-gotti-are-influencing-the-future-of-music-on-youtube-971db915
12 https://techcrunch.com/2023/06/26/adobe-indemnity-clause-designed-to-ease-enterprise-fears-about-ai-generated-art/
13 https://www.washingtonpost.com/world/2023/09/03/ai-regulation-law-china-israel-eu/
14 https://www.europarl.europa.eu/news/en/headlines/society/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence
15 https://www.bloomberg.com/news/articles/2023-09-12/ai-leaders-chatgpt-s-altman-call-for-more-us-tech-oversight-explained?leadSource=uverify%20wall
16 https://www.whitehouse.gov/briefing-room/statements-releases/2023/10/30/fact-sheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthy-artificial-intelligence/
17 https://www.nytimes.com/2023/08/16/technology/ai-gpu-chips-shortage.html
18 https://www.theinformation.com/articles/microsoft-rations-access-to-ai-hardware-for-internal-teams
19 https://www.cnbc.com/2021/10/16/tsmc-taiwanese-chipmaker-ramping-production-to-end-chip-shortage.html
20 https://www.cnbc.com/2023/09/18/morgan-stanley-chatgpt-financial-advisors.html
21 https://www.bloomberg.com/news/articles/2023-03-07/griffin-says-trying-to-negotiate-enterprise-wide-chatgpt-license
22 https://www.bloomberg.com/news/articles/2023-04-26/jpmorgan-s-ai-puts-25-years-of-federal-reserve-talk-into-a-hawk-dove-score
Please enter a valid e-mail address
Please enter a valid e-mail address
Important legal information about the e-mail you will be sending. By using this service, you agree to input your real e-mail address and only send it to people you know. It is a violation of law in some jurisdictions to falsely identify yourself in an e-mail. All information you provide will be used by Fidelity solely for the purpose of sending the e-mail on your behalf.The subject line of the e-mail you send will be "Fidelity.com: "

Your e-mail has been sent.

Your e-mail has been sent.

Related Articles

Artificial Intelligence
By: John Dalton | June 6, 2024
FCAT interviews Ethan Mollick, author of Co-Intelligence: Living and Working with AI about how AI will impact our personal and professional lives.
Artificial Intelligence
By: Sarah Hoffman | December 6, 2023
Sarah Hoffman, VP of AI and Machine Learning Research in FCAT speaks with Juliette Powell, a researcher, entrepreneur, and keynote speaker at the intersection of technology and business, about how organizations can embrace responsible technology.
Artificial Intelligence
By: Sarah Hoffman | November 29, 2023
Generative AI still shows great promise, but its transformational power for large enterprises has so far been limited. While there are numerous challenges, inaccuracy, cybersecurity, intellectual-property infringement, and regulatory compliance are the four most commonly cited risks that organizations are working to mitigate. The good news: solutions are coming that will make it easier for enterprises to move forward over the coming year.

This website is operated by Fidelity Center for Applied Technology (FCAT)® which is part of Fidelity Wealth Technologies, LLC, a Fidelity Investments company. FCAT experiments with and provides innovative products, services, content and tools, as a service to its affiliates and as a subsidiary of FMR LLC. Based on user reaction and input, FCAT is better able to engage in technology research and planning for the Fidelity family of companies. FCATalyst.com is independent of fidelity.com. Unless otherwise indicated, the information and items published on this web site are provided by FCAT and are not intended to provide tax, legal, insurance or investment advice and should not be construed as an offer to sell, a solicitation of an offer to buy, or a recommendation for any security by any Fidelity entity or any third-party. In circumstances where FCAT is making available either a product or service of an affiliate through this site, the affiliated company will be identified. Third party trademarks appearing herein are the property of their respective owners. All other trademarks are the property of FMR LLC.


This is for persons in the U.S. only.

245 Summer St, Boston MA

© 2008-2024 FMR LLC All right reserved | FCATalyst.com

Terms of Use | Privacy | Security | DAT Support