diff --git a/telephony-bot-api/fastfood-order-bot-example/README.md b/telephony-bot-api/fastfood-order-bot-example/README.md new file mode 100644 index 00000000..78fc9c91 --- /dev/null +++ b/telephony-bot-api/fastfood-order-bot-example/README.md @@ -0,0 +1,117 @@ +# Project Overview + +The Project is to create a fast food ordering voicebot using Langchain and Voicegain Telephony Bot API. +The bot has the ability to order from a predefined menu that includes burgers, wraps, drinks, fries and potato wedges. +The bot can distinguish between all vegetarian items and non-veg items. +The user has the ability to customize their order by adding topics / add-ons to their burger or wrap +and have the ability to pick from 3 different sizes for the fries and potato wedges. +They can further create custom combo orders and have a choice between 3 - main item with a drink, +main item with a side of their fries or potato wedges, main with a side and a drink. +If an order qualifies for a combo order - the bot automatically creates a combo for the user. +The bot can summarize the users order at any point along with calculating the total cost of the order. +The user can further edit their order at any point - by either changing an item (replace item, change quantity etc) +or deleting an item entirely. + +# Project Steps + +There are mainly 3 steps to it - +- Implement the Bot Logic +- Implement a Telephony Bot API service +- Setting up the AIVR bot on Voicegain Customer Portal. + +Each of these work hand in hand to create the bot. + +## 1. Bot Logic + +For the bot logic, we use LangChain which is a framework to build with LLMs by chaining interoperable components. +This helps in managing the workflow of the bot. + +For Langchain Reference kindly refer the Documentation here - https://python.langchain.com/v0.2/docs/introduction/ + +If the reader is interested in the detailed bot implementation, kindly read through --- + +The bot logic is powered by LangChain and OpenAI's API. +In order to create the bot framework, chains are used to link the various functions that the bot can perform. +By using tools, the program is able to create a basic function framework that ChatGPT can use depending on the task. +It is important to note that the function description is key as this informs ChatGPT +what the function of each of these functions are, and can respectively pick the appropriate function for the task. +This logic for bot is present in [fast_food_bot.py](fast_food_bot.py). + +The core functions are therefore created as tool functions in LangChain. +Functions such as giving the order summary and the total order cost have their own function as well, +so as to prevent possible errors arising from ChatGPT. + +### 1.1 Tools + +- get_menu: Returns the menu and prices for burgers, wraps, fries, wedges, and drinks. +- get_add_ons: Lists all possible add-ons and their costs. +- get_combos(main_item, side=None, drink=None): Provides possible meal combos based on user selections. +- update_order(new_order: OrderItem): Updates the current order, handles adding, updating, and deleting items. +- get_final_price: Calculates the total price of the current order. +- get_breakdown: Provides a detailed breakdown of the current order. + +### 1.2 Agents and Chains + +- ChatPromptTemplate: Defines the conversation prompt template. +- create_tool_calling_agent: Creates an agent that uses the defined tools and prompt. +- AgentExecutor: Manages the execution of the agent with tools. +- RunnableWithMessageHistory: Manages conversation history and processes the inputs + +### 1.3 Steps to run the bot locally - + +- Install the requirements in requirements.txt +- Set your openai api key in 'fast_food_bot.py' file. +- Run the main.py file to start the local uvicorn server. +- Run the http_client_example now to interact with the bot via text. + + +## 2. Telephony Bot API Integration + +Reference Document: https://console.voicegain.ai/api-documentation#tag/aivr-callback + +Implement POST / PUT / DELETE + +- POST: We create a new call session with unique 'csid'. They can be used to uniquely identify the call. +- PUT: If we get user output, call CHAIN to relay information to the bot. If not, reprompt the user for a response. +- DELETE: Delete the session from Chain's memory + +This is the component that integrates Voicegain's Telephony Bot API with the function of the langChain bot. +This adds the ability to interact with the bot through speech, rather than text, +thereby enhancing the user interaction experience. This logic is present in [main.py](main.py). + + +### 2.1 Functions + +- create_session(request: TelephonyBotPostReq): Adds a new call session with unique sid, csid. +- update_session(request: TelephonyBotPutReq): Updates an existing order using the sid, csid based on the user input. Forwards input to the Fast Food Bot and returns the response. +- delete_session(request: TelephonyBotDeleteReq): Removes the call session and associated chat history from the in-memory store by csid. + + +## 3. Create AIVR App on Voicegain Portal. + +By making an AIVR App on the Voicegain Portal, we can enable the speech component to our food ordering bot. This has mainly 3 steps - + +- Hosting Bot on a server +- Providing the URL in the AIVR App created on the Voicegain Portal + +We explore these steps in detail below. + +### 3.1 Hosting Bot on Server + +When you've run the [main.py](main.py) file and the bot is running locally on your device, you need to host the bot on a public URL for using on AIVR app. +One of the options for that is to use Ngrok - https://dashboard.ngrok.com/get-started/setup/windows + +### 3.2 Setting up AIVR App + +After you've configured the public URL for the food ordering bot, we need to setup the AIVR App using the following steps - + +1. Login to the Voicegain console - https://console.voicegain.ai/specific/aivr-apps +2. Buy a phone number in the phone management section. Click on buy number, add your area code and buy. This will be used later in the Main DNS field for AIVR App. +3. Now move to 'Phone Apps' section from 'Phone Management' in the toolbar on the left. Add new AIVR App using the blue button on top-right and filling in the details. Remember to add the number you purchased in the 'Main DNIS' section. +4. Click on Add Logic at the end and fill in details. The public URL where you host the bot goes in here +5. After that click on OK and Save to save your AIVR bot. You should get a screen like this with a number in Main DNIS field - + +Now you can call on the phone number in the Main DNIS and you can use the food ordering bot on call. + +### Credits - +Voicegain Team \ No newline at end of file diff --git a/telephony-bot-api/fastfood-order-bot-example/fast_food_bot.py b/telephony-bot-api/fastfood-order-bot-example/fast_food_bot.py new file mode 100644 index 00000000..e34e8657 --- /dev/null +++ b/telephony-bot-api/fastfood-order-bot-example/fast_food_bot.py @@ -0,0 +1,233 @@ +from langchain_openai import ChatOpenAI +from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder +from langchain_core.pydantic_v1 import BaseModel, Field +from langchain_core.chat_history import InMemoryChatMessageHistory +from langchain_core.runnables.history import RunnableWithMessageHistory +from langchain_core.tools import tool +from langchain.agents import AgentExecutor, create_tool_calling_agent +import os +from typing import List + +os.environ["OPENAI_API_KEY"] = 'YOUR-OPENAI-KEY' + +llm = ChatOpenAI(model="gpt-4o-mini") + + +# tool to contain the menu of the restaurant +@tool +def get_menu(): + """Get fast food restaurant menu and price for each item. The price is in dollars + the menu has burgers and wraps. There are also fries and wedges that can be ordered, and there are 3 different sizes for them as well.""" + menu = { + "burgers": [ + {"name": "Cheeseburger", "type": "non-veg", "price": 10}, + {"name": "Black Bean Burger", "type": "veg", "price": 12}, + {"name": "Hamburger", "type": "non-veg", "price": 12}, + {"name": "Crispy Chicken Burger", "type": "non-veg", "price": 12}, + {"name": "Veggie Burger", "type": "veg", "price": 12}, + {"name": "Fish Burger", "type": "non-veg", "price": 12}, + {"name": "BBQ Chicken Burger", "type": "non-veg", "price": 12}, + {"name": "Spicy Chicken Burger", "type": "non-veg", "price": 12}, + {"name": "Turkey Burger", "type": "non-veg", "price": 12}, + {"name": "Mushroom Swiss Burger", "type": "veg", "price": 12} + ], + "wraps": [ + {"name": "Grilled Chicken Wrap", "type": "non-veg", "price": 10}, + {"name": "Spicy Chicken Wrap", "type": "non-veg", "price": 10}, + {"name": "BBQ Chicken Wrap", "type": "non-veg", "price": 10}, + {"name": "Veggie Wrap", "type": "veg", "price": 10}, + {"name": "Turkey Wrap", "type": "non-veg", "price": 10}, + {"name": "Buffalo Chicken Wrap", "type": "non-veg", "price": 10}, + {"name": "Chicken Caesar Wrap", "type": "non-veg", "price": 10}, + {"name": "Steak Wrap", "type": "non-veg", "price": 10}, + {"name": "Falafel Wrap", "type": "veg", "price": 10}, + {"name": "Hummus Veggie Wrap", "type": "veg", "price": 10} + ], + "fries": [ + {"name": "Regular Fries", "sizes": {"Small": 2, "Medium": 3, "Large": 4}}, + {"name": "Curly Fries", "sizes": {"Small": 2.5, "Medium": 3.5, "Large": 4.5}}, + {"name": "Sweet Potato Fries", "sizes": {"Small": 3, "Medium": 4, "Large": 5}}, + {"name": "Waffle Fries", "sizes": {"Small": 3, "Medium": 4, "Large": 5}}, + {"name": "Cajun Fries", "sizes": {"Small": 2.5, "Medium": 3.5, "Large": 4.5}}, + {"name": "Chili Cheese Fries", "sizes": {"Small": 4, "Medium": 5, "Large": 6}}, + {"name": "Garlic Fries", "sizes": {"Small": 3, "Medium": 4, "Large": 5}}, + {"name": "Truffle Fries", "sizes": {"Small": 4.5, "Medium": 5.5, "Large": 6.5}}, + {"name": "Loaded Fries", "sizes": {"Small": 5, "Medium": 6, "Large": 7}}, + {"name": "Cheese Fries", "sizes": {"Small": 3.5, "Medium": 4.5, "Large": 5.5}} + ], + "wedges": [ + {"name": "Potato Wedges", "sizes": {"Small": 3, "Medium": 4, "Large": 5}}, + {"name": "Cajun Wedges", "sizes": {"Small": 3.5, "Medium": 4.5, "Large": 5.5}}, + {"name": "Garlic Parmesan Wedges", "sizes": {"Small": 4, "Medium": 5, "Large": 6}}, + {"name": "Buffalo Wedges", "sizes": {"Small": 4, "Medium": 5, "Large": 6}}, + {"name": "Loaded Potato Wedges", "sizes": {"Small": 5, "Medium": 6, "Large": 7}}, + {"name": "Cheesy Wedges", "sizes": {"Small": 3.5, "Medium": 4.5, "Large": 5.5}}, + {"name": "BBQ Wedges", "sizes": {"Small": 4, "Medium": 5, "Large": 6}}, + {"name": "Spicy Wedges", "sizes": {"Small": 3.5, "Medium": 4.5, "Large": 5.5}}, + {"name": "Truffle Wedges", "sizes": {"Small": 4.5, "Medium": 5.5, "Large": 6.5}}, + {"name": "Herb Wedges", "sizes": {"Small": 3.5, "Medium": 4.5, "Large": 5.5}} + ], + "drinks": [ + {"name": "Coca-Cola", "size": "Medium", "price": 2}, + {"name": "Pepsi", "size": "Medium", "price": 2}, + {"name": "Sprite", "size": "Medium", "price": 2}, + {"name": "Fanta Orange", "size": "Medium", "price": 2}, + {"name": "Dr. Pepper", "size": "Medium", "price": 2}, + {"name": "Mountain Dew", "size": "Medium", "price": 2}, + {"name": "Root Beer", "size": "Medium", "price": 2}, + {"name": "Iced Tea", "size": "Medium", "price": 2}, + {"name": "Lemonade", "size": "Medium", "price": 2}, + {"name": "Fruit Punch", "size": "Medium", "price": 2} + ] + } + return menu + + +# list of all add ons for the items +@tool +def get_add_ons(): + """all possible add ons """ + add_ons = [ + {"name": "Extra Cheese", "cost": "1"}, + {"name": "Lettuce", "cost": "1"}, + {"name": "Tomatoes", "cost": "1"}, + {"name": "Bacon", "cost": "1.5"}, + {"name": "Avocado", "cost": "2"}, + {"name": "Jalapenos", "cost": "0.5"}, + {"name": "Extra Patty", "cost": "3"}, + {"name": "Guacamole", "cost": "2"}, + {"name": "Pickles", "cost": "0.5"}, + {"name": "Onions", "cost": "1"}, + {"name": "Mushrooms", "cost": "1.5"}, + {"name": "BBQ Sauce", "cost": "0.5"}, + {"name": "Ranch Dressing", "cost": "0.5"}, + {"name": "Honey Mustard", "cost": "0.5"}, + {"name": "Chipotle Sauce", "cost": "0.5"}, + ] + return add_ons + + +# tool to help define what combos are possible +@tool +def get_combos(main_item: str, side: str = None, drink: str = None): + """Create a meal combo of the user's choice. The user can select a burger or wrap from the menu, + along with an optional drink and either fries or wedges with that as well, which is also optional.""" + combos = { + "Main with a Side": {"main_item": main_item, "side": side, "price": 10}, + "Main with a Drink": {"main_item": main_item, "drink": drink, "price": 11}, + "Main with Side and Drink": {"main_item": main_item, "side": side, "drink": drink, "price": 12} + } + return combos + + +class OrderItem(BaseModel): + item: str = Field(description="the item user ordered from the menu") + price: float = Field(description="price of the item") + delete: bool = Field(description="Whether the user want to delete this item") + quantity: int = Field(description="the amount of items ordered") + add_on: str = Field(description="the add on item ordered from the menu") + price_add_on: int = Field(description="the price of the add on items") + combined_cost: int = Field(description="the cost of the order including the items and the odd ons") + + +# to store the current order +my_order: List[OrderItem] = [] + + +@tool +def update_order(new_order: OrderItem): + """This function needs to be called when the user order something, or update their order, or delete something""" + global my_order + + if new_order.delete: + # Remove the item from the order + my_order = [item for item in my_order if not (item.item == new_order.item and item.add_on == new_order.add_on)] + else: + item_found = False + for item in my_order: + if item.item == new_order.item and item.add_on == new_order.add_on: + item_found = True + if new_order.quantity > 0: + item.quantity = new_order.quantity + item.price = new_order.price + item.price_add_on = new_order.price_add_on + item.combined_cost = item.price + item.price_add_on + else: + my_order.remove(item) + break + + if not item_found and new_order.quantity > 0: + my_order.append(new_order) + + print(f"Updated order: {my_order}") + + +@tool +def get_final_price(): + """ Get the final price of the order """ + global my_order + total_price = sum(item.combined_cost * item.quantity for item in my_order) + return total_price + + +# Function to get breakdown of the current order +@tool +def get_breakdown(): + """ function to get the cost breakdown of the order """ + global my_order + to_return = "" + total = 0 + for item in my_order: + to_return += f'{item.item} with {item.add_on}: cost {item.combined_cost}\n' + total += item.combined_cost + to_return += f'total order cost is {total}' + return to_return + + +tools = [get_menu, update_order, get_final_price, get_add_ons, get_combos, get_breakdown] + + +prompt = ChatPromptTemplate.from_messages( + [ + ( + "system", + "You are a helpful assistant for a fast food ordering service. Answer all questions to the best of your ability." + "You are a voice bot, so only return sentences without formatting. ", + ), + MessagesPlaceholder("chat_history"), + ("human", "{input}"), + ("placeholder", "{agent_scratchpad}"), + ] +) + +chain = prompt | llm + + +MEMORY_HISTORY_STORE = {} + + +# storing chat history +def get_session_history(session_id: str) -> InMemoryChatMessageHistory: + if session_id not in MEMORY_HISTORY_STORE: + MEMORY_HISTORY_STORE[session_id] = InMemoryChatMessageHistory() + return MEMORY_HISTORY_STORE[session_id] + + +agent = create_tool_calling_agent(llm, tools, prompt) + +agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True) + +CHAIN = RunnableWithMessageHistory( + agent_executor, + get_session_history, + input_messages_key="input", + history_messages_key="chat_history", + output_messages_key="output", +) + + +if __name__ == "__main__": + while True: + input_text = input("input: ") + output = CHAIN.invoke({"input": input_text}, config={'configurable': {'session_id': 'foo'}})["output"] + print(output) diff --git a/telephony-bot-api/fastfood-order-bot-example/http_client_example.py b/telephony-bot-api/fastfood-order-bot-example/http_client_example.py new file mode 100644 index 00000000..18970fab --- /dev/null +++ b/telephony-bot-api/fastfood-order-bot-example/http_client_example.py @@ -0,0 +1,49 @@ +import requests + +SID = "test-sid" +seq = 1 + +URL = "http://localhost:8000/bot" + + +# init post +response = requests.post( + URL, + json={ + "sid": SID, + "sequence": seq + } +).json() +init_prompt = response['question']['text'] +csid = response["csid"] + +print("Assistant: {}".format(init_prompt)) + +while True: + user_input = input("You: ") + seq += 1 + response = requests.put( + f"{URL}?seq={seq}", json={ + "sid": SID, + "csid": csid, + "events": [ + { + "sequence": str(seq), + "type": "input", + "timeMsec": 0, + "vuiResult": "MATCH", + "vuiAlternatives": [ + { + "utterance": user_input, + "confidence": 1.0 + } + ] + } + ] + + } + ).json() + print(response) + print("Assistant: {}".format(response['question']['text'])) + + diff --git a/telephony-bot-api/fastfood-order-bot-example/main.py b/telephony-bot-api/fastfood-order-bot-example/main.py new file mode 100644 index 00000000..c3cc7781 --- /dev/null +++ b/telephony-bot-api/fastfood-order-bot-example/main.py @@ -0,0 +1,85 @@ +from fastapi import FastAPI, HTTPException +from model import * +from fast_food_bot import CHAIN, MEMORY_HISTORY_STORE + +app = FastAPI() + + +@app.post("/bot") +async def create_session(request: TelephonyBotPostReq): + sid = request.sid + csid = f"bot-{sid}" + return TelephonyBotPostPutResp( + csid=csid, + sid=sid, + question=Question( + text="Thank you for calling Big Burger. What can I do for you today?" + ) + ) + + +@app.put("/bot") +async def update_session(request: TelephonyBotPutReq): + csid = request.csid + sid = request.sid + bot_answer = None + user_input = None + + if request.events: + for event in request.events: + if isinstance(event, InputEvent): + if event.vuiResult == "MATCH": + if event.vuiAlternatives: + user_input = event.vuiAlternatives[0].utterance + else: + print(f"Get vuiResult {event.vuiResult}") + break + + if user_input: + bot_answer = CHAIN.invoke( + {"input": user_input}, + config={'configurable': {'session_id': csid}} + )["output"] + + if bot_answer is None: + return TelephonyBotPostPutResp( + csid=csid, + sid=sid, + question=Question( + text="Sorry, I didn't get that. Can you say it again?" + ) + ) + else: + return TelephonyBotPostPutResp( + csid=csid, + sid=sid, + question=Question( + text=bot_answer + ) + ) + + +@app.delete("/bot") +async def delete_session(request: TelephonyBotDeleteReq): + csid = request.csid + result = MEMORY_HISTORY_STORE.pop(csid, None) + if result: + return TelephonyBotDeleteResp( + csid=csid, + termination="Success" + ) + else: + return TelephonyBotDeleteResp( + csid=csid, + termination="Session not found" + ) + + +@app.get("/bot") +async def root(): + return {"message": "Welcome to the Fast Food Ordering Service"} + + +if __name__ == "__main__": + import uvicorn + uvicorn.run(app, host="0.0.0.0", port=8000) diff --git a/telephony-bot-api/fastfood-order-bot-example/model.py b/telephony-bot-api/fastfood-order-bot-example/model.py new file mode 100644 index 00000000..56a89f59 --- /dev/null +++ b/telephony-bot-api/fastfood-order-bot-example/model.py @@ -0,0 +1,55 @@ +from pydantic import BaseModel +from typing import Optional, List, Union + + +class Prompt(BaseModel): + text: str + + +class Question(BaseModel): + text: str + + +class VuiAlternatives(BaseModel): + utterance: str + + +class InputEvent(BaseModel): + vuiResult: str + vuiAlternatives: Optional[List[VuiAlternatives]] = None + + +class OtherEvent(BaseModel): + pass + + +# Request model for POST +class TelephonyBotPostReq(BaseModel): + sid: str + + +# Request model for PUT +class TelephonyBotPutReq(BaseModel): + csid: str + sid: str + events: Optional[List[Union[InputEvent, OtherEvent]]] = None + + +# Request model for DELETE +class TelephonyBotDeleteReq(BaseModel): + csid: str + sid: str + + +# Response model for POST and PUT request +class TelephonyBotPostPutResp(BaseModel): + csid: str + sid: str + prompt: Optional[Prompt] = None + question: Optional[Question] = None + + +# Response model for DELETE request +class TelephonyBotDeleteResp(BaseModel): + csid: str + termination: str diff --git a/telephony-bot-api/fastfood-order-bot-example/requirements.txt b/telephony-bot-api/fastfood-order-bot-example/requirements.txt new file mode 100644 index 00000000..24432047 --- /dev/null +++ b/telephony-bot-api/fastfood-order-bot-example/requirements.txt @@ -0,0 +1,5 @@ +langchain_openai +langchain_core +langchain +fastapi +uvicorn \ No newline at end of file diff --git a/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-1.png b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-1.png new file mode 100644 index 00000000..ae97703b Binary files /dev/null and b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-1.png differ diff --git a/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-2.png b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-2.png new file mode 100644 index 00000000..2b52cc42 Binary files /dev/null and b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-2.png differ diff --git a/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-3.png b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-3.png new file mode 100644 index 00000000..0b6673ae Binary files /dev/null and b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-3.png differ diff --git a/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-4.png b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-4.png new file mode 100644 index 00000000..b2eb1813 Binary files /dev/null and b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-4.png differ diff --git a/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-5.png b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-5.png new file mode 100644 index 00000000..b3f7ba4c Binary files /dev/null and b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-5.png differ diff --git a/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-6.png b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-6.png new file mode 100644 index 00000000..e3d0795b Binary files /dev/null and b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-6.png differ diff --git a/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-7.png b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-7.png new file mode 100644 index 00000000..b1fc6695 Binary files /dev/null and b/telephony-bot-api/fastfood-order-bot-example/screenshots/Screenshot-7.png differ