How to ChatGPT

Spring into AI: Must-Know Insights about ChatGPT

ChatGPT was released to the public more than a year ago, yet many users continue to hold misconceptions about its capabilities and use. This article addresses some of these misunderstandings, and teaches you to wield ChatGPT as a valuable tool. Here are 5 points you must know to live with ChatGPT and similar products. (11m read)

1. It is a word prediction engine.

At its core, ChatGPT is based on something called a large language model (LLM). You can think of the LLM as a gigantic database that holds the probabilities of how words follow other words and how sets of words follow each other. ChatGPT and other chatbots use this information to string together words in a process called text generation. Because the database holds information about how words are combined by humans, the text that is generated from those probabilities sounds remarkably human. We sometimes say it is the world's best word prediction engine.

Two Pillars of Quality

To understand what ChatGPT is doing, it is important to know that these models, like most AI, are based on two pillars of 

1) data it was fed from the internet

and

2) statistical models the systems learned during “training”.

(1) The data consists of words from millions or billions of web pages and databases that have been scraped from the internet to document how humans combine words in language.

(2) The statistics consist of the probabilities that some words follow others and how sets of words are likely to occur together, calculated from those documents. This section has been simplified for purposes of this article, read more about LLMs here.

The data pillar leads to the fact that the language generation is better in areas where there is a lot of data and worse when there is less data. The probability pillar leads to the fact that it does a better job when probabilities of sequences are high and consistent and the developers had a lot of money to build a complex model.

There are more implications as well. For example, because words are generated based on probabilities of sequences, the final result is never perfectly predictable. You will often get different results for the same request (or prompt), and the quality may vary as well. On the data side, the responses will depend on the types of data it was trained on. In some systems, lots of training data came from Reddit, which is an online question and answer forum. Like society in general, some of the conversations are polite with factually correct answers, and some, not so much. The underlying model is trained to mimic how people talk and communicate, but in some subject areas, it may not have the ability to distinguish the fully factual, scientific view of a topic.

Additionally, it is important to know that the underlying LLMs of ChatGPT (GPT 3.5 or GPT 4) were trained on data from a particular time period. GPT 3.5, the model behind the original ChatGPT released in November 2022, has a training data cutoff of September 2021. The current version of GPT 4, available to subscription users of ChatGPT Plus, was trained on data up until April 2023. Thus, when describing current concepts, the model has less accurate responses due to the decreased language available to it after its training cutoff. For example, programming languages may have been revised, new studies have been published, and current events have greatly shifted. Despite ChatGPT Plus users having access to real-time data from the internet, when prompted with the subject of current events, the program will typically preface its answer by acknowledging this limitation. This data limitation applies to all of its abilities. For example, if a prompt asks ChatGPT to write a specific form of computer code, it may generate code with outdated methods or terms that no longer exist. If it asks for restaurants in the area, ChatGPT may output a list that includes a business that has since closed. Keeping this in mind will help you perceive the bot’s accuracy and know when it is time to consult other resources.

2. It doesn’t “know” anything, but can say a lot of true and wonderful things.

Fluency Matters

Part of the amazing feel of ChatGPT is that the text generated is incredibly human sounding. In fact, it can be so natural sounding that it can be disorienting because this is the first widespread instance of a machine doing this before. Sometimes, part of this experience is that the fluency, speed, and smoothness of the language generated is so human sounding, it may be disarming and lead to loosening guards on accuracy checks.

Some things it is really good at:

1. Computer Programming. While it is great at human language, it is really best at computer languages. This is because computer languages are typically much more predictable than human language. Many experienced, professional computer programmers incorporate ChatGPT into their workflow, a practice that is augmenting the typical programmer’s workflow and rapidly changing the industry.

While one might think this is irrelevant if they are not a computer programmer, it is important to know that computer programming encompasses tasks such as writing scripts and functions for Google Sheets or Microsoft Excel. ChatGPT is typically quite good at these tasks because it has been fed a lot of data of this type. ChatGPT is also very good at debugging. Paste the questionable code in and ask for help finding the bug, or error. Just make sure you know how to evaluate and confirm the code is doing what is truly expected and that it is not hallucinating, or making things up.

2. Writing in a specific genre. Many unwanted and mundane tasks, like writing job descriptions, for example, have been fundamentally changed by the advent of ChatGPT. Writing job applications is often not a desired task, and people often go hunting for examples to copy from and edit. Well, ChatGPT has already looked at millions of example job descriptions and can do much of this work itself. Make sure to tell it that you want it to write a job description for a job with the following characteristics and list them in as detailed a manner as you can. Additionally, make sure to read it intently and edit it when it is complete.

It can do lots of other things based on its knowledge of genres: Press releases, marketing materials, short stories, recipes, travel itineraries, lists, etc. Whatever you need to get done that involves human language, give ChatGPT a try. For example, if you are wondering how to write a letter of recommendation, ask it for a template or give it the information as background.

3. Summarization. In general, ChatGPT is quite good at summarization. Give it a piece of writing and ask for a summary. If you are good at computer coding (or can get ChatGPT to do it), you could write code to automate summarization across documents through the ChatGPT API (which allows a computer program to use the brains of ChatGPT to do some tasks repeatedly). There has also been success using ChatGPT to categorize written research responses into categories. Of course, check that you truly believe the results and vary the number of categories to see different results.

4. Making lists and generating ideas. ChatGPT is a great brainstorming tool. Whether you are trying to think of what to cook for dinner, what to include in your marketing campaign, or what topic to write your essay on, the program can help spark your creativity by offering ideas. Further, it can make lists of what you may need to focus on or how to organize your thoughts. While human imagination is key, ChatGPT can push individuals to think of new ideas, shifting the starting point.

5. Serving as a writing aid. In addition to thinking of ideas for writing, ChatGPT can also function as a writing assistant. It can help create and organize your headings, rephrase any sentences that don’t sound quite right, and edit for grammar. Think of ChatGPT as a peer editor–it can provide helpful suggestions that may polish your writing. Notably, ChatGPT was used in this way for a portion of this article.

3. It has good intent but is not trustworthy for important tasks.

Let's get back to “probably,” as in “probability.” As we noted above, ChatGPT does many amazing language-based tasks that appear alarmingly human…except when it doesn’t.

Unfortunately, when you ask the system to do something for which it has little data, it may continue with its language generation in a way that humans consider “not true.” Computer scientists call this “hallucinating,” and while the marketing folks are still trying to rebrand it as “helpful mistakes,” it can be a serious problem if you are counting on the answer to help you do an important task.

There are widespread discussions about the rate of hallucination, how to decrease it in future models, and when it occurs. Occurrence seems to be most associated with our two pillars. On the data side, if you ask it for something for which it needs to respond but does not have data, it will keep generating away by stringing together words that are sensible in form, but not necessarily true in our reality. For example, when I ask it to “tell me about Ken Kennedy” it tells me there could be multiple Ken Kennedys and I should disambiguate. When I specify Ken Kennedy from real estate development (a field chosen at random), it typically says some very nice things about the person, including references to nonexistent cities or completely made-up projects. There aren’t millions of web pages about the fictional Ken Kennedy to use for word prediction. However, it has been trained on millions of biographies and papers regarding real estate development, so it has plenty to generate in general.

When we combine this with the probability issues, we have a trust problem. You don’t know when you can trust it and when you cannot. What to do? That depends. If you are writing poems or fiction, hallucinate away. It might be great! If you want technical advice for something someone else will use, make sure you are 100% capable of evaluating the quality of the response. ChatGPT is a prediction model, not a genius. So, while ChatGPT can wrap its digital brain around a user's input and spit out suitable responses, it's simply predicting outcomes based on training data rather than genuine encounters. This means that the model has only a limited grasp of human emotions, cultural nuances, and personal experiences. Sure, it can perform some critical analysis, but it can't make sense of it all in the same way we can.

4. ChatGPT Loves Context.

How do you reduce the probability of hallucinations and make your experience with ChatGPT effective? Be more deliberate about how you prompt it. Go above and beyond how you would typically craft a Google search and think about how to best procure the result you’re looking for. This is known as prompt engineering, and it focuses on designing and optimizing prompts to generate a desired outcome. But one doesn't need to be a professional prompt engineer to know how to elevate the use of ChatGPT.

1. Context is key: When crafting prompts, tell ChatGPT specific objectives, an audience, and any other context you deem necessary. While the bot can actively learn during your conversation, it won’t know who you are or what you want unless you tell it.

2. Give it a persona: Ask it to generate content as if it were a specific persona. Think of it as if you are complimenting the bot, and tell it how good it is at whatever task you’re asking it to complete.

3. Use unconventional prompts: If you’re looking for unique or unexpected responses that unlock the creative potential of this technology, prompts that are more complex than a simple Google search are the way to go.

4. Ask ChatGPT to vary its output: ChatGPT may only respond in text (and recently, images for certain users), but that doesn’t mean it can’t have a format! Some ideas you can try are asking it to respond tabularly, with a mind map, with an outline, in a specific essay format (eg: persuasive), at a specific length, or any kind of structure you want.

5. No need to be short and sweet: Prompts don’t need to be concise or complete sentences. Make sure to build out your prompts–the more ChatGPT knows, the more effectively it can respond.

6. Keep chatting: ChatGPT can adjust as it interacts with you, so if the first response isn’t what you wanted—keep going. Ask follow up questions, point out errors, and continue to guide it in your desired direction.

5. Experimentation is Key.

The last insight to getting in tune with ChatGPT is to know that experimentation is key. The technology is constantly changing and evolving, and best practices change with it. New strategies are being researched daily. To find what works best for you and your personal goals, you should experiment, trying out different methods of prompt engineering, task usage, and evaluation. In playing with ChatGPT in this way, you will likely find many insights of your own.

Conclusion

ChatGPT has taken the world by storm through its impressive capabilities, from passing the bar exam to solving medical mysteries to generating APIs with code. While companies race to apply ChatGPT to different applications and develop new features, it certainly will continue to evolve our education, work, and digital interactions. As ChatGPT (and similar models and products) propel us into the future, it is crucial to be deliberate and thoughtful about how we can get the most out of this technology, fostering a nuanced approach as we all partake in the world of AI’s unfolding possibilities.

***all imagery created using Image Creator from Designer***

The New AI Project | University of Notre Dame

Editors: Grace Hatfield, Graham Wolfe

Contributors: John Behrens, Grace Hatfield, Sydney Colgan, Rachel Lee

Advisor: John Behrens