It works surprisingly well. Results of ChatGPT are very good – even better than GPT-3, which, while released by OpenAI back in 2020, was still the best general purpose language model (See Figure 1). And, even with so many users, the results are generally returned within seconds. It also retains more context than we’ve seen before. While GPT-3 has a context length of 2,048 tokens (a token is approximately 4 characters in English), ChatGPT’s context length is double that. A chatbot’s long-term memory is important, given that it makes conversing with it – especially a long conversation – much easier (see Figure 2).

It’s openly available. Anyone can use ChatGPT; you can check it out at chat.openai.com. It was released for free for the first two months but there is now a fee for premium plans. GPT-3, by comparison, was released with a waitlist for a year and a half before being opened to anyone in supported countries willing and able to pay for it. Moreover, ChatGPT is no-code, meaning there’s no need to know anything about machine learning to use it. In fact, it’s even more accessible than GPT-3, which has numerous controls to understand.

Will This Change Search?

The release of ChatGPT inevitably raises the question: what about Google? Indeed, the release of ChatGPT led Google’s management to declare a “code red”. Alphabet also announced a new chatbot technology Bard that the company will begin rolling out in the coming weeks.² It’s easy to understand why. A side-by-side comparison of the search experience shows just how “dated” Google’s search experience already feels (see Figure 3). Not only is the definition of ChatGPT clearer and easier to understand coming from ChatGPT than Google search, but you can also make specifications like “explain this in 100 characters”, “explain this to a high school student”, or even ask it to write its reply in the style of a specific writer. All of this surpasses what can be done by Google today. In fact, Microsoft, which has invested billions in OpenAI, already began rolling out a version of its Bing search that uses the AI behind ChatGPT.³

But that doesn’t mean Google search is done for. While ChatGPT is impressive, it has serious shortcomings that will prevent this version from knocking Google off its perch. “Wow” factor aside, ChatGPT is hardly a reliable search or research tool, suffering from:

Misinformation. Misinformation is a big problem when it comes to language models. StackOverflow temporarily banned answers generated by GPT and ChatGPT, since the answer often looks correct but actually contains mistakes, and users who posted these answers weren’t verifying that they were fully correct before posting them.⁴ Language models today also don’t give you their sources, so solely relying on them for anything factual is unwise.

Limited understanding. Language models are incapable of comprehending the meaning of the data they are processing and can therefore make mistakes answering even simple questions (see Figure 4). Their understanding of time is limited too. Ask ChatGPT for new technology trends, and you may get trends that were being discussed five years ago (and ChatGPT doesn’t know anything past 2021).

Why It Matters

When a leading tech company like Google goes code red, it’s worth taking notice. And while we don’t think ChatGPT will displace Google search anytime soon, this first release gives us a glimpse into how AI will impact our lives in the not-too-distant future not by replacing us, but by helping us with a range of cognitive tasks. Already we can imagine a tool like this assisting with:

Upgrading our approach to financial literacy and corporate training. Financial literacy has always been a challenge, but as the financial world gets even more complicated – with new assets, new currencies, and new frameworks – it’s time to think differently about financial education. While we may not want these tools to advise customers directly, perhaps they can help educate employees, and even help customers prepare before they meet with representatives, so they may be able to ask better questions coming in. One huge benefit of AI chatbots for education: there is no shame in feeling uneducated. People can ask any questions to the AI, even very basic questions, without having to feel like someone may be judging them (see Figure 5).

Augmenting brainstorming efforts. As a brainstorming tool, chatbots like ChatGPT can be very useful. Whether brainstorming new product ideas, marketing campaigns, companies to invest in, or ways to use an emerging technology, language model technology could be part of the brainstorming process. In these contexts, where the point is to find new ideas or spark provocative conversations, the verifiability of the answer is not as important as the novelty of the insight. Here, ChatGPT is already a useful brainstorming assistant (see Figure 6).

Generating drafts for written communication. Whether it’s financial reports, emails to customers, website copy, or even job descriptions, tools like ChatGPT can help us make the writing process more efficient by writing first drafts for us or even editing our own writing (see Figure 7).

Signals To Watch

Improved interfaces. The big signal to look out for is integration of Google-like capabilities with ChatGPT. One small vendor to watch: Ought, makers of Elicit, an AI-powered research tool that produces reliable, verifiable results. While it’s not as conversational as ChatGPT, we believe it's headed in the right direction. If the future of search is closer to ChatGPT’s vision of an interface than to Google’s current offering, it’s at the interface layer where this battle will be won (remember the powerful simplicity of Google’s search box when it first shipped?)

Scandal. It’s inevitable that some student, author, publisher will abuse these tools and raise the specter of AIs gone wild. Already a professor had one student admit to using it to write an essay.⁵ Definitely expect more stories like this to come to light, especially if services like ChatGPT remain on freemium models and are easily accessible.

Industry-specific integration. More industry-specific language models are coming. Above, we mentioned Elicit, which is focused on helping with academic research. Another research tool powered by GPT-3 that is making waves: Scispace.⁶ Meta’s Galactica, while only around for a few days before they removed the demo because of a backlash due to misinformation, was specifically focused on scientific research.⁷ A month later, at the end of 2022, Google announced Med-PaLM, a large language model that generates answers to medical questions.⁸

Acceptance of the unexplainable. One of the challenges with more advanced AIs like ChatGPT is that even the researchers who’ve built the engine can’t explain how the software generates specific results. So much of what we call “knowledge” is explainable, and we depend on experts to be able to readily understand the how and why behind the what. But with AI, we don’t always understand the how, but the what is useful, even powerful. Researchers recently used a deep neural network to analyze daytime satellite imagery in an effort to predict future economic growth.⁹ The results were significantly more accurate than any alternative traditional mathematical model. How will accepting black-box algorithms change our relationship to knowledge work and knowledge workers? Is understanding overrated?