04_Chapter 4. Examples and Few-Shot Prompting.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **Chapter 4. Examples and Few-Shot Prompting** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a number of well-known prompting techniques that use examples, such as zero-shot, one-shot, and few-shot prompting. By understanding each approach and the principles of designing effective examples, you can significantly enhance the performance of Solar. \n",
    "\n",
    "This chapter is based on Prompt `Type C`:  Instruction + Context + **Example** + Output "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **Table of Contents**\n",
    "- Use `Ctrl + F` (Windows) or `Cmd + F` (Mac) to locate specific sections by title."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **4.1 Zero-Shot Prompting**\n",
    "\n",
    "    - 4.1.1 Introduction\n",
    "\n",
    "    - 4.1.2 Example\n",
    "\n",
    "    - 4.1.3 Practice\n",
    "\n",
    "\n",
    "- **4.2 One-Shot Prompting**\n",
    "\n",
    "    - 4.2.1 Introduction\n",
    "\n",
    "    - 4.2.2 Example\n",
    "\n",
    "    - 4.2.3 Practice\n",
    "\n",
    "\n",
    "- **4.3 Few-Shot Prompting**\n",
    "\n",
    "    - 4.3.1 Introduction\n",
    "\n",
    "    - 4.3.2 Examples\n",
    "\n",
    "        - (1) Exemplar Quantity\n",
    "\n",
    "        - (2) Exemplar Similarity\n",
    "\n",
    "        - (3) Exemplar Format"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Set up**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success!\n"
     ]
    }
   ],
   "source": [
    "from openai import OpenAI\n",
    "\n",
    "# Retrieve the UPSTAGE_API_KEY variable from the IPython store\n",
    "%store -r UPSTAGE_API_KEY\n",
    "\n",
    "try:\n",
    "    if UPSTAGE_API_KEY:\n",
    "        print(\"Success!\")\n",
    "except NameError as ne:\n",
    "    print(f\"Since, {ne}\")\n",
    "    print(\"Please, insert your API key.\")\n",
    "    UPSTAGE_API_KEY = input(\"UPSTAGE_API_KEY =\")\n",
    "\n",
    "# Set your API key: \n",
    "# UPSTAGE_API_KEY = \" \" ←- Insert your API key here.\n",
    "\n",
    "client = OpenAI(\n",
    "    api_key= UPSTAGE_API_KEY,\n",
    "    base_url=\"https://api.upstage.ai/v1/solar\"\n",
    ")\n",
    "\n",
    "config_model = {\n",
    "    \"model\": \"solar-pro\",\n",
    "    \"max_tokens\": 2000,\n",
    "    \"temperature\": 0.7,\n",
    "    \"top_p\": 0.9,\n",
    "}\n",
    "\n",
    "def get_completion(messages, system_prompt=\"\", config=config_model):\n",
    "    try:\n",
    "        if system_prompt:\n",
    "            messages = [{\"role\": \"system\", \"content\": system_prompt}] + messages\n",
    "\n",
    "        message = client.chat.completions.create(messages=messages, **config)\n",
    "        return message.choices[0].message.content\n",
    "    \n",
    "    except Exception as e:\n",
    "        print(f\"Error during API call: {e}\")\n",
    "        return None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **4.1 Zero-Shot Prompting**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **4.1.1 Introduction**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Zero-shot prompting** is a technique where the model is asked to perform a task without any prior examples. There are numerous ways to craft zero-shot prompts. This is one of the powerful features of language models—it allows them to perform various tasks with minimal information."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **4.1.2 Example**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sentiment: Neutral \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Prompt:\n",
    "message = [\n",
    "    {\n",
    "        \"role\":\"user\",\n",
    "        \"content\": \"Classify the following text as positive, negative, or neutral. Text: I thought the macaron flavor was just okay. Sentiment: { }\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **4.1.3 Practice**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "→ Translate the term 'artificial tears' from English into Japanese. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prompt:\n",
    "message = [\n",
    "    {\n",
    "        \"role\":\"user\",\n",
    "        \"content\": \" \" # ←- Insert your prompt here.\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **4.2 One-Shot Prompting**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **4.2.1 Introduction**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**One-shot prompting** is a technique where the model is provided with only one example before it’s asked to complete the task."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **4.2.2 Example**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "J'adore étudier les langues. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Prompt:\n",
    "message = [\n",
    "    {\n",
    "        \"role\":\"user\",\n",
    "        \"content\": \"Translate the following sentence into French: I like reading books.\"\n",
    "    },\n",
    "    {\n",
    "        \"role\":\"assistant\",\n",
    "        \"content\": \"J'aime lire des livres.\"\n",
    "    },\n",
    "    {\n",
    "        \"role\":\"user\",\n",
    "        \"content\": \"Translate the following sentence into French: I love studying language\"\n",
    "    },\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **4.2.3 Practice**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "→ Translate the term 'artificial tears' from English to Japanese using one-shot prompting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prompt:\n",
    "message = [\n",
    "    {\" \"} # ←- Insert your prompt here.\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **4.3 Few-Shot Prompting**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **4.3.1 Introduction**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Few-shot prompting** is a technique where the model is given a handful of examples—typically two to five—before it is asked to complete the task. This approach offers the model multiple references for the type of output desired, providing more context and making the output more accurate or aligned with specific patterns."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> It's important to note that **few-shot learning** and **few-shot prompting** are distinct concepts. Few-shot learning is a broader machine learning approach focused on adapting model parameters with only a few examples. In contrast, **few-shot prompting** specifically applies to prompt design in generative AI, where model parameters remain unchanged."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **4.3.2 Examples**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " four hundred and one \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Prompt:\n",
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"2+10:\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"assistant\",\n",
    "        \"content\": \"twelve\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"4+52:\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"assistant\",\n",
    "        \"content\": \"fifty-six\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"100+301:\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We would like to highlight **three ways** to design effective examples when using Solar:\n",
    "\n",
    "- **Exemplar Quantity**\n",
    "- **Exemplar Similarity**\n",
    "- **Exemplar Format**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **(1) Exemplar Quantity**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mostly, the quantity of exemplars in the prompt generally improves model\n",
    "performance. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sentence: \"Thick smog chokes northern India and eastern Pakistan ahead of Diwali.\"\n",
      "Subject: \"Thick smog\"\n",
      "Verb: \"chokes\"\n",
      "Direct Object: \"northern India and eastern Pakistan\"\n",
      "Prepositional Phrase: \"ahead of Diwali\"\n",
      "Analysis: Subject-verb-direct object structure with a prepositional phrase providing additional context. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Prompt:\n",
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"The cat sleeps.\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"assistant\",\n",
    "        \"content\": \"\"\"Sentence: \"The cat sleeps.\"\n",
    "Subject: \"The cat\"\n",
    "Verb: \"sleeps\"\n",
    "Object: None\n",
    "Analysis: Simple subject-verb sentence structure with a single actor performing an action.\"\"\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"She reads books.\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"assistant\",\n",
    "        \"content\": \"\"\"Sentence: \"She reads books.\"\n",
    "Subject: \"She\"\n",
    "Verb: \"reads\"\n",
    "Object: \"books\"\n",
    "Analysis: Subject-verb-object structure, indicating the action is directed toward an object.\"\"\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"They dance gracefully.\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"assistant\",\n",
    "        \"content\": \"\"\"Sentence: \"They dance gracefully.\"\n",
    "Subject: \"They\"\n",
    "Verb: \"dance\"\n",
    "Adverb: \"gracefully\"\n",
    "Analysis: Subject-verb-adverb structure, with the adverb modifying how the action is performed.\"\"\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"The sun rises in the east.\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"assistant\",\n",
    "        \"content\": \"\"\"Sentence: \"The sun rises in the east.\"\n",
    "Subject: \"The sun\"\n",
    "Verb: \"rises\"\n",
    "Prepositional Phrase: \"in the east\"\n",
    "Analysis: Subject-verb-prepositional phrase structure, adding context about the action's location.\"\"\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"Thick smog chokes northern India and eastern Pakistan ahead of Diwali.\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "In the sentence \"Thick smog chokes northern India and eastern Pakistan ahead of Diwali,\" the components are as follows:\n",
      "\n",
      "1. Thick smog: Subject (noun phrase), referring to the polluted air.\n",
      "2. chokes: Verb, indicating the action of causing difficulty in breathing.\n",
      "3. northern India and eastern Pakistan: Direct object (noun phrase), referring to the locations affected by the smog.\n",
      "4. ahead of Diwali: Prepositional phrase, providing context and indicating the time of the event.\n",
      "\n",
      "The sentence describes a situation where the polluted air, or smog, is causing difficulty in breathing in northern India and eastern Pakistan, and this situation occurs before the Diwali festival. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# without an example\n",
    "\n",
    "# Prompt:\n",
    "message = [\n",
    "    {\n",
    "        \"role\":\"user\",\n",
    "        \"content\": \"\"\"Please analyze and define the sentence components in the following sentence.\n",
    "        sentence: \"Thick smog chokes northern India and eastern Pakistan ahead of Diwali.\"\n",
    "        \"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Observation: If you provide an example, you can see the model respond in the format used in the example. Even with complex and varied sentence structures, using many examples to guide the model helps it perform the task well. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **(2) Exemplar Similarity**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Select exemplars that are similar to your task. For example, if you are summarizing a news article, using exemplars in the same format will yield the best results you desire."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following prompt example, you can see that the summary results use **a consistent response format** as an example. By increasing the consistency of the format, you can achieve more accurate results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "#OSAID / Defines open source AI as a model that allows full understanding of its construction and provides usage rights like the freedom to use, modify, and build on top./\n",
      "#Maffulli / Criticizes Meta's use of \"open source\" label and expects the AI community to correct misuse of the term./\n",
      "#AI Study / Finds that many \"open source\" models are not truly open source, as they keep training data secret, require significant compute power, and use complex fine-tuning techniques./ \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# When examples have the similarities\n",
    "# Prompt:\n",
    "message = [\n",
    "    {\n",
    "        \"role\":\"user\",\n",
    "        \"content\": \"\"\"Read the following text and summarize the key points. Once you finish the summary, organize it as follows:\n",
    "\n",
    "#Keyword 1 / Related key point in one short phrase./\n",
    "#Keyword 2 / Related key point in one short phrase./\n",
    "#Keyword 3 / Related key point in one short phrase./\n",
    "\n",
    "<text>\n",
    "To be considered open source under the OSAID, an AI model has to provide enough information about its design so that a person could “substantially” recreate it. The model must also disclose any pertinent details about its training data, including the provenance, how the data was processed, and how it can be obtained or licensed.\n",
    "“An open source AI is an AI model that allows you to fully understand how it’s been built,” Maffulli said. “That means that you have access to all the components, such as the complete code used for training and data filtering.”\n",
    "The OSAID also lays out usage rights developers should expect with open source AI, like the freedom to use the model for any purpose and modify it without having to ask anyone’s permission. “Most importantly, you should be able to build on top,” added Maffulli.\n",
    "The OSI has no enforcement mechanisms to speak of. It can’t pressure developers to abide by or follow the OSAID. But it does intend to flag models described as “open source” but which fall short of the definition.\n",
    "“Our hope is that when someone tries to abuse the term, the AI community will say, ‘We don’t recognize this as open source,’ and it gets corrected,” Maffulli said. Historically, this has had mixed results, but it isn’t entirely without effect.\n",
    "Many startups and big tech companies, most prominently Meta, have employed the term “open source” to describe their AI model release strategies — but few meet the OSAID’s criteria. For example, Meta mandates that platforms with over 700 million monthly active users request a special license to use its [Llama](https://techcrunch.com/2024/09/08/meta-llama-everything-you-need-to-know-about-the-open-generative-ai-model/) models.\n",
    "Maffulli has been [openly critical](https://www.ft.com/content/397c50d8-8796-4042-a814-0ac2c068361f) of Meta’s decision to call its models “open source.” After discussions with the OSI, Google and Microsoft agreed to drop their use of the term for models that aren’t fully open, but Meta hasn’t, he said.\n",
    "Stability AI, which has long advertised its models as “open,” requires that businesses making more than $1 million in revenue obtain an enterprise license. And French AI upstart Mistral’s license bars the use of certain models and outputs for commercial ventures.\n",
    "A [study](https://www.wired.com/story/the-myth-of-open-source-ai/) last August by researchers at the Signal Foundation, the nonprofit AI Now Institute, and Carnegie Mellon found that many “open source” models are basically open source in name only. The data required to train the models is kept secret, the compute power needed to run them is beyond the reach of many developers, and the techniques to fine-tune them are intimidatingly complex.\n",
    "Instead of democratizing AI, these “open source” projects tend to entrench and expand centralized power, the study’s authors concluded. Indeed, Meta’s Lllama models have [racked up](https://venturebeat.com/ai/meta-leads-open-source-ai-boom-llama-downloads-surge-10x-year-over-year/) hundreds of millions of downloads, and Stability [claims](https://www.prnewswire.com/news-releases/stability-ai-secures-significant-new-investment-from-world-class-investor-group-and-appoints-prem-akkaraju-as-ceo-302181923.html) that its models power up to 80% of all AI-generated imagery.\n",
    "</text>\"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# When examples don't have the similarities\n",
    "# Prompt:\n",
    "message = [\n",
    "    {\n",
    "        \"role\":\"user\",\n",
    "        \"content\": \"\"\"Read the following text and summarize the key points. Once you finish the summary, organize it as follows:\n",
    "\n",
    "Related key point in one short phrase./Keyword 1\n",
    "Keyword 2 Related key point in two short phrases.\n",
    "Keyword 3/ Related key point in three short phrases.\n",
    "\n",
    "<text>\n",
    "To be considered open source under the OSAID, an AI model has to provide enough information about its design so that a person could “substantially” recreate it. The model must also disclose any pertinent details about its training data, including the provenance, how the data was processed, and how it can be obtained or licensed.\n",
    "“An open source AI is an AI model that allows you to fully understand how it’s been built,” Maffulli said. “That means that you have access to all the components, such as the complete code used for training and data filtering.”\n",
    "The OSAID also lays out usage rights developers should expect with open source AI, like the freedom to use the model for any purpose and modify it without having to ask anyone’s permission. “Most importantly, you should be able to build on top,” added Maffulli.\n",
    "The OSI has no enforcement mechanisms to speak of. It can’t pressure developers to abide by or follow the OSAID. But it does intend to flag models described as “open source” but which fall short of the definition.\n",
    "“Our hope is that when someone tries to abuse the term, the AI community will say, ‘We don’t recognize this as open source,’ and it gets corrected,” Maffulli said. Historically, this has had mixed results, but it isn’t entirely without effect.\n",
    "Many startups and big tech companies, most prominently Meta, have employed the term “open source” to describe their AI model release strategies — but few meet the OSAID’s criteria. For example, Meta mandates that platforms with over 700 million monthly active users request a special license to use its [Llama](https://techcrunch.com/2024/09/08/meta-llama-everything-you-need-to-know-about-the-open-generative-ai-model/) models.\n",
    "Maffulli has been [openly critical](https://www.ft.com/content/397c50d8-8796-4042-a814-0ac2c068361f) of Meta’s decision to call its models “open source.” After discussions with the OSI, Google and Microsoft agreed to drop their use of the term for models that aren’t fully open, but Meta hasn’t, he said.\n",
    "Stability AI, which has long advertised its models as “open,” requires that businesses making more than $1 million in revenue obtain an enterprise license. And French AI upstart Mistral’s license bars the use of certain models and outputs for commercial ventures.\n",
    "A [study](https://www.wired.com/story/the-myth-of-open-source-ai/) last August by researchers at the Signal Foundation, the nonprofit AI Now Institute, and Carnegie Mellon found that many “open source” models are basically open source in name only. The data required to train the models is kept secret, the compute power needed to run them is beyond the reach of many developers, and the techniques to fine-tune them are intimidatingly complex.\n",
    "Instead of democratizing AI, these “open source” projects tend to entrench and expand centralized power, the study’s authors concluded. Indeed, Meta’s Lllama models have [racked up](https://venturebeat.com/ai/meta-leads-open-source-ai-boom-llama-downloads-surge-10x-year-over-year/) hundreds of millions of downloads, and Stability [claims](https://www.prnewswire.com/news-releases/stability-ai-secures-significant-new-investment-from-world-class-investor-group-and-appoints-prem-akkaraju-as-ceo-302181923.html) that its models power up to 80% of all AI-generated imagery.\n",
    "\n",
    "</text>\"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "responses = []\n",
    "for i in range(3):\n",
    "    response = get_completion(messages=message)\n",
    "    responses.append(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- As seen in the following results, prompts with lower similarity among examples yield a wider variety of outcomes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "n = 1\n",
      "Open source AI criteria/Keyword 1\n",
      "Keyword 2 Disclosure of AI model design and training data/Keyword 3\n",
      "Keyword 4 Freedom to use, modify, and build upon open source AI models/Keyword 5\n",
      "\n",
      "\n",
      "n = 2\n",
      "Open source AI model definition/OSAID\n",
      "Maffulli explains Open source AI as a model with full understanding of its build, including access to all components, such as complete code for training and data filtering.\n",
      "OSI's role/OSI enforcement\n",
      "The OSI aims to flag models described as \"open source\" but falling short of the definition, with no enforcement mechanisms, relying on the AI community to correct misuse.\n",
      "Study on \"open source\" AI models/Study findings\n",
      "A study found that many \"open source\" models are open source in name only, with secret training data, inaccessible compute power, and complex fine-tuning techniques, entrenching centralized power.\n",
      "\n",
      "\n",
      "n = 3\n",
      "Open source AI definition/OSAID requires access to complete code, training data, and usage rights.\n",
      "OSI enforcement/OSI has no enforcement mechanisms but can flag models that don't meet the definition.\n",
      "AI community's response/AI community may reject misuse of term, but results are mixed.\n",
      "\n",
      "Many companies use \"open source\" label inaccurately/Few companies meet OSAID criteria, like Meta's special license requirement for large platforms.\n",
      "Stability AI and Mistral impose licensing restrictions/These companies require enterprise licenses or restrict commercial use.\n",
      "Study reveals limitations of \"open source\" AI/Many \"open source\" models are complex, require significant compute power, and have hidden data.\n",
      "Open source AI projects entrench centralized power/These projects often benefit large companies, like Meta and Stability AI.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for i in range(3):\n",
    "    print(f\"n = {i+1}\")\n",
    "    print(responses[i])\n",
    "    print(\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **(3) Exemplar Format**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use a common template. The optimal format may vary across tasks. \n",
    "\n",
    "There is evidence indicating that formats frequently found in the training data tend to result in improved performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "- company names: \n",
      "- personal names: \n",
      "- specific topics: \n",
      "    - SOLAR 10.7B\n",
      "    - large language model (LLM)\n",
      "    - depth up-scaling (DUS)\n",
      "    - mixture-of-experts\n",
      "    - natural language processing (NLP)\n",
      "    - instruction-following capabilities\n",
      "- themes: \n",
      "    - language model development\n",
      "    - model scaling\n",
      "    - model efficiency\n",
      "    - model fine-tuning\n",
      "    - open-source licensing \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Prompt:\n",
    "message = [\n",
    "    {\n",
    "        \"role\":\"user\",\n",
    "        \"content\": \"\"\"Identify the entities referenced in the text below. \n",
    "\n",
    "Please extract the following four types of entities:\n",
    "- company names: \n",
    "- personal names: \n",
    "- specific topics:\n",
    "- themes:\n",
    "\n",
    "<text>\n",
    "We introduce SOLAR 10.7B, a large language model (LLM) with 10.7 billion parameters, demonstrating superior performance in various natural language processing (NLP) tasks. Inspired by recent efforts to efficiently up-scale LLMs, we present a method for scaling LLMs called depth up-scaling (DUS), which encompasses depthwise scaling and continued pretraining. In contrast to other LLM up-scaling methods that use mixture-of-experts, DUS does not require complex changes to train and inference efficiently. We show experimentally that DUS is simple yet effective in scaling up high-performance LLMs from small ones. Building on the DUS model, we additionally present SOLAR 10.7B-Instruct, a variant fine-tuned for instruction-following capabilities, surpassing Mixtral-8x7B-Instruct. SOLAR 10.7B is publicly available under the Apache 2.0 license, promoting broad access and application in the LLM field.\n",
    "</text>\"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# When not using a standard format\n",
    "# Prompt:\n",
    "message = [\n",
    "    {\n",
    "        \"role\":\"user\",\n",
    "        \"content\": \"\"\"Identify the entities referenced in the text below. \n",
    "\n",
    "Please extract the following four types of entities:\n",
    "% % company names\n",
    "% % personal names \n",
    "% % specific topics% % \n",
    "% %  themes % % \n",
    "\n",
    "~text~\n",
    "We introduce SOLAR 10.7B, a large language model (LLM) with 10.7 billion parameters, demonstrating superior performance in various natural language processing (NLP) tasks. Inspired by recent efforts to efficiently up-scale LLMs, we present a method for scaling LLMs called depth up-scaling (DUS), which encompasses depthwise scaling and continued pretraining. In contrast to other LLM up-scaling methods that use mixture-of-experts, DUS does not require complex changes to train and inference efficiently. We show experimentally that DUS is simple yet effective in scaling up high-performance LLMs from small ones. Building on the DUS model, we additionally present SOLAR 10.7B-Instruct, a variant fine-tuned for instruction-following capabilities, surpassing Mixtral-8x7B-Instruct. SOLAR 10.7B is publicly available under the Apache 2.0 license, promoting broad access and application in the LLM field.\n",
    "~text~\"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "responses = []\n",
    "for i in range(2):\n",
    "    response = get_completion(messages=message)\n",
    "    responses.append(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- When using unusual format symbols or marks, the stability of the results decreases each time they are generated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "n = 1\n",
      "company names: SOLAR\n",
      "\n",
      "personal names:\n",
      "\n",
      "specific topics:\n",
      "\n",
      "themes: Large language models (LLMs), natural language processing (NLP), scaling LLMs, depth up-scaling (DUS), mixture-of-experts, instruction-following capabilities, Apache 2.0 license\n",
      "\n",
      "\n",
      "n = 2\n",
      "company names: SOLAR\n",
      "\n",
      "personal names:\n",
      "\n",
      "specific topics: language model (LLM), natural language processing (NLP), depth up-scaling (DUS), mixture-of-experts, instruction-following capabilities\n",
      "\n",
      "themes: Large language model performance, LLM scaling methods, LLM accessibility\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for i in range(2):\n",
    "    print(f\"n = {i+1}\")\n",
    "    print(responses[i])\n",
    "    print(\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**References**\n",
    "\n",
    "[Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., ... & Amodei, D. (2020). *Language models are few-shot learners.* Advances in Neural Information Processing Systems, 33, 1877-1901.](https://arxiv.org/abs/2005.14165)\n",
    "\n",
    "[Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., ... & Le, Q. V. (2021). *Finetuned language models are zero-shot learners.* arXiv preprint arXiv:2109.01652.](https://arxiv.org/abs/2109.01652)\n",
    "\n",
    "[Work, W. M. I. C. L. *Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?*](https://arxiv.org/abs/2202.12837)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Next: [Chapter 5. Role and Style Prompting](./05_Chapter%205.%20Role%20and%20Style%20Prompting.ipynb)*"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}