08_Chapter 8. Resolving Hallucinations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **Chapter 8. Resolving Hallucinations**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Solar sometimes produces **hallucinations**—instances where the language model generates factually incorrect, fabricated, or irrelevant information—undermining the reliability of AI outputs, especially when accuracy is essential. \n",
    "\n",
    "The best way to resolve hallucinations in LLMs is using the technique named Retrieval-Augmented Generation(RAG), which integrates knowledge sources to provide more accurate and contextually relevant responses. In this cookbook, we do not cover RAG, but we introduce methods to mitigate hallucinations in Solar at the prompt level. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This chapter introduces two methods to reduce or prevent hallucinations. \n",
    "\n",
    "- **Method 1. Distinguishing Between Facts and Opinions**\n",
    "\n",
    "- **Method 2. I don’t Know for Unknown Information**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **Table of Contents**\n",
    "- Use `Ctrl + F` (Windows) or `Cmd + F` (Mac) to locate specific sections by title."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **8.1 Distinguishing Between Facts and Opinions**\n",
    "\n",
    "\n",
    "- **8.2 `Use References; Say \"I don't know\"`**\n",
    "\n",
    "\n",
    "- **8.3 Practice**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Set up**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success!\n"
     ]
    }
   ],
   "source": [
    "from openai import OpenAI\n",
    "\n",
    "# Retrieve the UPSTAGE_API_KEY variable from the IPython store\n",
    "%store -r UPSTAGE_API_KEY\n",
    "\n",
    "try:\n",
    "    if UPSTAGE_API_KEY:\n",
    "        print(\"Success!\")\n",
    "except NameError as ne:\n",
    "    print(f\"Since, {ne}\")\n",
    "    print(\"Please, insert your API key.\")\n",
    "    UPSTAGE_API_KEY = input(\"UPSTAGE_API_KEY =\")\n",
    "\n",
    "# Set your API key: \n",
    "# UPSTAGE_API_KEY = \" \" ←- Insert your API key here. \n",
    "\n",
    "client = OpenAI(\n",
    "    api_key= UPSTAGE_API_KEY,\n",
    "    base_url=\"https://api.upstage.ai/v1/solar\"\n",
    ")\n",
    "\n",
    "config_model = {\n",
    "    \"model\": \"solar-pro\",\n",
    "    \"max_tokens\": 996,\n",
    "    \"temperature\": 0.5,\n",
    "    \"top_p\": 1.0,\n",
    "}\n",
    "\n",
    "def get_completion(messages, system_prompt=\"\", config=config_model):\n",
    "    try:\n",
    "        if system_prompt:\n",
    "            messages = [{\"role\": \"system\", \"content\": system_prompt}] + messages\n",
    "\n",
    "        message = client.chat.completions.create(messages=messages, **config)\n",
    "        return message.choices[0].message.content\n",
    "    \n",
    "    except Exception as e:\n",
    "        print(f\"Error during API call: {e}\")\n",
    "        return None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **8.1 Distinguishing Between Facts and Opinions**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a sentence where Solar produces a hallucination."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The original distance of a marathon is 26.2 miles, or 42.195 kilometers. This distance was standardized in 1921 and is based on the legendary run of a Greek soldier named Pheidippides, who is said to have run from the battlefield of Marathon to Athens to deliver news of a victory. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Imitative Falsehood\n",
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"What is the original distance of a marathon?\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although the standard distance of a marathon is 42.195 kilometers (26.22 miles), it is based on the route from the start at Windsor Castle to the royal entrance of the White City Stadium. The original distance of the marathon was approximately 40 kilometers (24.85 miles, or around 25 miles). The model’s response reflects a common misconception present in its training data.\n",
    "\n",
    "When large language models (LLMs) are trained on inaccurate data, they may unintentionally reinforce these errors, resulting in factually incorrect outputs, often referred to as '**imitative falsehoods**' ([Lin et al., 2022](https://arxiv.org/abs/2109.07958))."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To reduce hallucinations in response to the question, \"*What is the original distance of a marathon?*\", the prompt can be structured to separate `'fact' and 'your opinion'` in the answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-The Fact: The original distance of a marathon is approximately 25 miles or 42 kilometers. This distance is based on the legend of Pheidippides, a Greek messenger who ran from the city of Marathon to Athens to announce the Greeks' victory over the Persians in the Battle of Marathon in 490 BC.\n",
      "\n",
      "-Your Opinion: In my opinion, the original distance of a marathon is a significant and iconic distance in the world of running and athletics. It represents a great challenge and accomplishment for runners, and the history and tradition behind it add to its allure and prestige. However, it's important to note that the actual distance may have varied over time and may not have been precisely measured. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"\"\"What is the original distance of a marathon?\n",
    "\n",
    "Make a clear distinction between the fact and your opinion. \n",
    "\n",
    "-The Fact: \n",
    "-Your Opinion: \"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we will tackle a more complicated task. We will provide an article to Solar and ask questions about it. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a question: What are the environmental impacts mentioned in the article regarding the early release of seasonal products like the PSL?\n",
    "\n",
    "**Article’s Source**: [The First Falling Leaf and a Psychologist’s Explanation for the Pumpkin Spice Latte Craze](https://www.inc.com/nick-hobson/the-first-falling-leaf-a-psychologists-explanation-for-pumpkin-spice-latte-craze.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "article = \"\"\"The First Falling Leaf and a Psychologist's Explanation for the Pumpkin Spice Latte Craze\n",
    "It signals a fresh start. And it's Starbucks' biggest success story.\n",
    "\n",
    "As the first leaf falls, signaling the subtle shift from summer to autumn, something stirs within us. It's not just the weather that changes; it's our entire psychological and emotional landscape. This transition has spawned a cultural phenomenon: the Pumpkin Spice Latte(PSL) craze. Today, on August 22nd, Starbucks released the PSL earlier than ever before. It's a day earlier than last year, but the sentiment is the same: It signals a natural shift in our consumer preferences.\n",
    "\n",
    "The psychological impact of seasonal changes\n",
    "The end of August and the beginning of September marks a significant psychological shift. Summer holidays wind down, children go back to school, and workplaces ramp up their efforts for the final quarter of the year. These changes are more than just logistical; they deeply influence our emotions, behavior, and interactions. Psychologically, this period represents a fresh stat-a concept well-documented in research. Known as the “fresh start effect,” this phenomenon suggests that people are more motivated to pursue goals and adopt new behaviors at the start of a new period, whether it's a week, a month, or a season.\n",
    "The fresh start effect could be seen as a powerful driver behind the PSL craze. As the days grow shorter and the air cooler, we instinctively seek comfort and familiarity. The PSL, with its warm spices and association with cozy autumn days, provides that comfort. It's a ritual that signals the beginning of a new season, much like the first falling leaf.\n",
    "\n",
    "The role of marketers in harnessing natural periods of change\n",
    "Marketers and brand strategists have long understood the power of seasonal changes to influence consumer behavior. The PSL is a prime example of how a product can become synonymous with a season, creating a sense of anticipation and urgency that drives sales. But there's a delicate balance to be struck. As Starbucks has released the PSL earlier and earlier each year, there's a risk that the product could lose its association with that \"Augtober\" timing, diluting its appeal. The earlier release may satisfy immediate demand, but it also raises the question: Are we pushing too hard against the natural rhythms that make seasonal products so special? My own research on rituals has shown that the best rituals are those that aren't messed with or changed. So, perhaps, Starbucks should pick a date at the end of August, the beginning of September, and stick with it.\n",
    "Successful marketers understand that the timing of a product launch is as crucial as the product itself. By aligning product releases with natural or culturally significant transitions, they can tap into the fresh start effect, creating a sense of newness and relevance. However, pushing too far ahead of these natural changes can backfire, leading to consumer fatigue and diminishing returns. The PSL's success lies in its ability to capture the essence of autumn, and if that connection is lost, the product may lose its allure.\n",
    "\n",
    "As the first leaf falls this year, and the scent of pumpkin spice wafts through the air earlier than ever, it's worth considering the psychological and emotional forces at play. The PSL is more than just a drink. It's a signal of change, a comforting ritual, and a testament to the power of seasons to shape our lives. Marketers and business leaders, take note: In a world that's constantly changing, celebrating the moments of transition is not just good business strategy, it's human culture at its finest.\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The article mentions two environmental impacts related to the early release of seasonal products like the PSL. First, it discusses the potential risk of diluting the product's appeal if it is released too early, which could lead to consumer fatigue and diminishing returns. Second, the article suggests that pushing too hard against natural rhythms can backfire, implying that maintaining a balance between product launches and natural transitions is crucial for long-term success. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"\"\"Q: What are the environmental impacts mentioned in the article regarding the early release of seasonal products like the PSL? \n",
    "-- \n",
    "{article}\n",
    "\"\"\".format(article=article)\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The provided article does not mention environmental impacts related to the early release of seasonal products like the PSL. Instead, it focuses on the psychological effects of seasonal transitions on consumer behavior and marketing strategies."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This issue can be resolved by requesting Solar to provide **evidence first and then answer**. The prompt should extract the most relevant and accurate sentences from the given text."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous example, we used a prompt to distinguish between 'facts' and 'your opinion,' but this time, we used a similar prompt to distinguish between `'the evidence you found' and 'your final answer.'` By doing this, when Solar is instructed to find evidence, it can locate information, which helps reduce the rate of hallucinations in longer texts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No environmental impacts are mentioned in the article regarding the early release of seasonal products like the PSL. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"\"\"What are the environmental impacts mentioned in the article regarding the early release of seasonal products like the PSL?  \n",
    "Find the most accurate and relevant sentence in the given text. \n",
    "Observe the response format:\n",
    "\n",
    "- Evidence You Found:  \n",
    "- Your Final Answer:  \n",
    "\n",
    "--\n",
    "<Text>\n",
    "{article}\n",
    "</Text>\n",
    "\"\"\".format(article=article)\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Remember what we learned in Chapter 6: for longer documents or texts, it’s better to use a structured prompt."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **8.2 `Use References; Say \"I don't know\"`**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To prevent hallucinations, ensure Solar responds with \"`I don't know`\" for unknown information when asked about current or up-to-date details after its knowledge cut-off. Alternatively, provide the current date and year before prompting an answer or supply Solar with the relevant information you wish to inquire about to help mitigate hallucinations. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here’s another example: Solar may produce hallucinations when it lacks data on recent events."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The most recent summer Olympics were held in Tokyo, Japan in 2021. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"Which city hosted the most recent summer Olympics?\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We believed that the most recent Summer Olympics took place in Paris, France, in 2024. This occurs because LLMs retain outdated information, illustrating a time-shift phenomenon in their responses. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I don't know. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# This problem can be solved as followed:\n",
    "\n",
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"\"\"Which city hosted the most recent summer Olympics? It is now the year 2024.\n",
    "        If you don't have updated information, please ONLY respond with 'I don't know.' \"\"\" \n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The most recent winner of the Nobel Prize in Literature is Annie Ernaux. She was announced as the recipient in 2023 for her work \"autobiographical narrative that gives the inherent worth of the human experience its rightful place.\" \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Let us try another example.\n",
    "\n",
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"Who most recently won the Nobel Prize in Literature?\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We knew the fact that the 2024 Nobel Prize in Literature was awarded to South Korean author Han Kang for her intense poetic prose that confronts historical traumas and exposes the fragility of human life."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can resolve this issue using two methods. One is to use an *‘I don’t know’* prompt and, the other is to add additional information to the prompt."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Method #1. I don't Know**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I don't know. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"\"\"Who most recently won the Nobel Prize in Literature in 2024? \n",
    "It is now the year 2024. If you don't have updated information, please ONLY respond with 'I don't know.'\"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Method #2. Additional Relevant Information**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The most recent winner of the Nobel Prize in Literature is Han Kang. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"\"\"Who most recently won the Nobel Prize in Literature?  \n",
    "Respond based on the following information: \n",
    "Han Kang is the first South Korean writer and the first Asian woman to receive the Nobel Prize in Literature in 2024. \n",
    "Her works are celebrated for their poetic and experimental style, addressing complex themes such as trauma, violence, and the human condition. \n",
    "This recognition highlights the growing global appreciation for Korean literature and culture, further elevating South Korea's presence in the international literary community.\"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **8.3 Practice**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise #1: Create a prompt that prevents hallucinations for the following questions**\n",
    "\n",
    "Who was elected president in the 2024 U.S. presidential election?  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"\"\" \"\"\" # ←- Insert your prompt here.\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise #2: Read the following Starbucks PSL article and write a prompt that answers the following question accurately**\n",
    "\n",
    "Question: How has the earlier release of the PSL affected consumer behavior, according to detailed statistical data? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"\"\"  Text: {article}\"\"\" # ←- Insert your prompt here.\n",
    "\n",
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": prompt.format(article=article) \n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Answer: The text does not provide numerical data on how the earlier release of the PSL has affected consumer behavior."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the best methods for resolving hallucinations is RAG (Retrieval-Augmented Generation), which grounds the LLMs during generation by conditioning on relevant documents retrieved from an external knowledge source. As this cookbook is highly focused on prompt engineering only, we will address some prompt engineering lessons in later chapters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Reference**\n",
    "\n",
    "Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., ... & Liu, T. (2023). *A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.* arXiv preprint arXiv:2311.05232."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Next: [Chapter 9. Complex Prompt Chaining](./09_Chapter%209.%20Complex%20Prompt%20Chaining.ipynb)*"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}