line_of_thought_organized.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AIML Engineer Take-Home Project\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise Info and Requirements"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Objective\n",
    "You are tasked with building a recommendation system using a provided public dataset,\n",
    "focusing on both classical recommendation models and modern LLM-based approaches.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Project Requirements\n",
    "Package your application as a service using a suitable framework, e.g. FastAPI or Flask. We\n",
    "are intentionally not being prescriptive here, but do your best to demonstrate your\n",
    "understanding of best practices when building your solution.\n",
    "1. Dataset\n",
    "  - Use the [Anime Recommendations Database from Kaggle](https://www.kaggle.com/datasets/CooperUnion/anime-recommendations-database). This dataset contains\n",
    "information on various anime, including user ratings, genres, and other relevant\n",
    "attributes.\n",
    "2. Data Endpoint\n",
    "  - Your service should have an API endpoint to query the dataset\n",
    "3. Classical Recommendation System\n",
    "  - Build a recommendation system that utilizes the dataset to suggest top k anime\n",
    "for a user based on their viewing history and preferences.\n",
    "  - Ensure that the recommendation logic excludes recently viewed items (e.g.,\n",
    "anime watched within the last 7 days).\n",
    "4. Contextual LLM-Based Personalization\n",
    "  - Implement a feature where users can get personalized anime recommendations\n",
    "based on a natural language description of their current mood or preferences\n",
    "(e.g., \"I want something uplifting and adventurous\").\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Submission\n",
    "Submit a single .zip file that includes:\n",
    "- all source code\n",
    "- A System Design doc\n",
    "\n",
    "  - Provide a document (e.g. `SYSTEM_DESIGN.md`) that explains your choices and\n",
    "the architecture.\n",
    "  - Discuss how you would extend the current system to make it more accurate in\n",
    "response to more “vague” user input\n",
    "  - Include recommendations of how to transition this to an LLM deployed in-house\n",
    "- A Presentation Slide Deck to present your project for 30 mins in during a panel interview\n",
    "  - Can include information from the System Design doc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install pandas\n",
    "# !pip install numpy\n",
    "# # !pip install seaborn\n",
    "# !pip install scikit-learn\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### First ideas\n",
    "\n",
    "My first idea is to use `NMF`, but let's first find out the current state of the art (SOTA) for recommender systems.\n",
    "\n",
    "#### Current state of the art\n",
    "\n",
    "There is not a single best algorithm for recommender systems. Solutions for the Netflix Prize include[^1][^2]:\n",
    "- Decomposition models (SVD, NMF, SVD++, etc.)\n",
    "- RBM\n",
    "- Decision Tree-based methods (Gradient Boosted Decision Trees, etc.)\n",
    "- Neural Networks\n",
    "- SVM\n",
    "\n",
    "Also, it is common to use a blend of models, like BellKor Solution to the Netflix Grand Prize[^3], which makes sense as many models tend to improve with ensemble methods.\n",
    "\n",
    "#### Way to go\n",
    "\n",
    "Considering this and the escope of this project, I will test simple models.\n",
    "\n",
    "\n",
    "[^1]: Stephen Gower. [Netflix Prize and SVD](http://buzzard.ups.edu/courses/2014spring/420projects/math420-UPS-spring-2014-gower-netflix-SVD.pdf). April 18th 2014 \n",
    "\n",
    "[^2]: [Netflix Recommendations: Beyond the 5 stars (Part 2)](https://netflixtechblog.com/netflix-recommendations-beyond-the-5-stars-part-2-d9b96aa399f5)\n",
    "\n",
    "[^3]: [The BellKor Solution to the Netflix Grand Prize](https://www2.seas.gwu.edu/~simhaweb/champalg/cf/papers/KorenBellKor2009.pdf)\n",
    "\n",
    "Considering this and the escope of this project, I will use the NMF algorithm."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature engineering\n",
    "\n",
    "Some ideas of features to be used in the model:\n",
    "- User's movie ratings\n",
    "- User's movie watching history\n",
    "- User's movie type history\n",
    "- User's movie episode size\n",
    "- Movie's average rating\n",
    "- Movie's popularity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import numpy as np\n",
    "# import pandas as pd\n",
    "# import matplotlib.pyplot as plt\n",
    "# from sklearn.metrics import mean_squared_error\n",
    "# from sklearn.model_selection import train_test_split\n",
    "# from sklearn.pipeline import Pipeline\n",
    "# from sklearn.linear_model import LinearRegression\n",
    "\n",
    "from utils import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "RATINGS_PATH = 'data/rating.csv'\n",
    "ANIME_PATH = 'data/anime.csv'\n",
    "ratings, anime = preprocess_data(RATINGS_PATH, ANIME_PATH)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NMF components (w1 and h1) have been saved to 'data/nmf_components.pkl'\n"
     ]
    }
   ],
   "source": [
    "rated = ratings[ratings.rating != -1]\n",
    "w1, h1, user_item_matrix = nmf(rated.iloc[0:10000], redo=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "user = np.int64(127)\n"
     ]
    }
   ],
   "source": [
    "user = rated.iloc[0:10000].sample(1)['user_id'].values[0]\n",
    "rec_anime_id, rec_weight = recommend(user, w1, h1, user_item_matrix)\n",
    "print(f'{user = }')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 3, 4]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = [1,2,3,4]\n",
    "a[:4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                        name  anime_id\n",
      "288               Fairy Tail      6702\n",
      "382   Kamisama Hajimemashita     14713\n",
      "440               Soul Eater      3588\n",
      "804         Sword Art Online     11757\n",
      "850             Gakuen Alice        74\n",
      "904                Special A      3470\n",
      "1046       Sukitte Ii na yo.     14289\n",
      "1083           Inu x Boku SS     11013\n",
      "1266            Shugo Chara!      2923\n",
      "1566          Kaze no Stigma      1691\n",
      "6702 288    Fairy Tail\n",
      "Name: name, dtype: object\n",
      "2923 1266    Shugo Chara!\n",
      "Name: name, dtype: object\n",
      "3588 440    Soul Eater\n",
      "Name: name, dtype: object\n",
      "11757 804    Sword Art Online\n",
      "Name: name, dtype: object\n",
      "3470 904    Special A\n",
      "Name: name, dtype: object\n",
      "14289 1046    Sukitte Ii na yo.\n",
      "Name: name, dtype: object\n",
      "74 850    Gakuen Alice\n",
      "Name: name, dtype: object\n",
      "1691 1566    Kaze no Stigma\n",
      "Name: name, dtype: object\n",
      "11013 1083    Inu x Boku SS\n",
      "Name: name, dtype: object\n",
      "14713 382    Kamisama Hajimemashita\n",
      "Name: name, dtype: object\n"
     ]
    }
   ],
   "source": [
    "print(anime[anime['anime_id'].isin(rec_anime_id[:10])][['name', 'anime_id']])\n",
    "for idx in rec_anime_id[:10]:\n",
    "    print(idx, anime[anime['anime_id'] == idx]['name'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "123\n"
     ]
    }
   ],
   "source": [
    "group_anime = recommend(user, w1, h1, user_item_matrix)\n",
    "print(user)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1,  2,  3,  5,  7,  8,  9, 10, 11])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rated.iloc[0:1000]['user_id'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "rec_anime_id, rec_weight = top_n_anime(user, group_anime, user_item_matrix, n=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(np.int64(123),\n",
       " Index([11061,    20,  8408,  6746, 11737, 22135, 13677, 12189,   895, 12231,\n",
       "        ...\n",
       "        10391, 10396, 10397, 10448, 10456, 10464, 10465, 10491, 10495,     1],\n",
       "       dtype='int64', name='anime_id', length=1717),\n",
       " array([0.26428815, 0.20498794, 0.19741301, ..., 0.        , 0.        ,\n",
       "        0.        ]))"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user, rec_anime_id, rec_weight\n",
    "# rec_cols = np.argsort(group_anime)[::-1]\n",
    "# group_anime[rec_cols] / np.sum(group_anime[rec_cols])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>anime_id</th>\n",
       "      <th>name</th>\n",
       "      <th>genre</th>\n",
       "      <th>type</th>\n",
       "      <th>episodes</th>\n",
       "      <th>rating</th>\n",
       "      <th>members</th>\n",
       "      <th>Action</th>\n",
       "      <th>Adventure</th>\n",
       "      <th>Cars</th>\n",
       "      <th>...</th>\n",
       "      <th>Slice of Life</th>\n",
       "      <th>Space</th>\n",
       "      <th>Sports</th>\n",
       "      <th>Super Power</th>\n",
       "      <th>Supernatural</th>\n",
       "      <th>Thriller</th>\n",
       "      <th>Unknown</th>\n",
       "      <th>Vampire</th>\n",
       "      <th>Yaoi</th>\n",
       "      <th>Yuri</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>32281</td>\n",
       "      <td>Kimi no Na wa.</td>\n",
       "      <td>Drama, Romance, School, Supernatural</td>\n",
       "      <td>Movie</td>\n",
       "      <td>1</td>\n",
       "      <td>9.37</td>\n",
       "      <td>200630</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5114</td>\n",
       "      <td>Fullmetal Alchemist: Brotherhood</td>\n",
       "      <td>Action, Adventure, Drama, Fantasy, Magic, Mili...</td>\n",
       "      <td>TV</td>\n",
       "      <td>64</td>\n",
       "      <td>9.26</td>\n",
       "      <td>793665</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28977</td>\n",
       "      <td>Gintama°</td>\n",
       "      <td>Action, Comedy, Historical, Parody, Samurai, S...</td>\n",
       "      <td>TV</td>\n",
       "      <td>51</td>\n",
       "      <td>9.25</td>\n",
       "      <td>114262</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>9253</td>\n",
       "      <td>Steins;Gate</td>\n",
       "      <td>Sci-Fi, Thriller</td>\n",
       "      <td>TV</td>\n",
       "      <td>24</td>\n",
       "      <td>9.17</td>\n",
       "      <td>673572</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>9969</td>\n",
       "      <td>Gintama&amp;#039;</td>\n",
       "      <td>Action, Comedy, Historical, Parody, Samurai, S...</td>\n",
       "      <td>TV</td>\n",
       "      <td>51</td>\n",
       "      <td>9.16</td>\n",
       "      <td>151266</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>32935</td>\n",
       "      <td>Haikyuu!!: Karasuno Koukou VS Shiratorizawa Ga...</td>\n",
       "      <td>Comedy, Drama, School, Shounen, Sports</td>\n",
       "      <td>TV</td>\n",
       "      <td>10</td>\n",
       "      <td>9.15</td>\n",
       "      <td>93351</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>11061</td>\n",
       "      <td>Hunter x Hunter (2011)</td>\n",
       "      <td>Action, Adventure, Shounen, Super Power</td>\n",
       "      <td>TV</td>\n",
       "      <td>148</td>\n",
       "      <td>9.13</td>\n",
       "      <td>425855</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>820</td>\n",
       "      <td>Ginga Eiyuu Densetsu</td>\n",
       "      <td>Drama, Military, Sci-Fi, Space</td>\n",
       "      <td>OVA</td>\n",
       "      <td>110</td>\n",
       "      <td>9.11</td>\n",
       "      <td>80679</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>15335</td>\n",
       "      <td>Gintama Movie: Kanketsu-hen - Yorozuya yo Eien...</td>\n",
       "      <td>Action, Comedy, Historical, Parody, Samurai, S...</td>\n",
       "      <td>Movie</td>\n",
       "      <td>1</td>\n",
       "      <td>9.10</td>\n",
       "      <td>72534</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>15417</td>\n",
       "      <td>Gintama&amp;#039;: Enchousen</td>\n",
       "      <td>Action, Comedy, Historical, Parody, Samurai, S...</td>\n",
       "      <td>TV</td>\n",
       "      <td>13</td>\n",
       "      <td>9.11</td>\n",
       "      <td>81109</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 51 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   anime_id                                               name  \\\n",
       "0     32281                                     Kimi no Na wa.   \n",
       "1      5114                   Fullmetal Alchemist: Brotherhood   \n",
       "2     28977                                           Gintama°   \n",
       "3      9253                                        Steins;Gate   \n",
       "4      9969                                      Gintama&#039;   \n",
       "5     32935  Haikyuu!!: Karasuno Koukou VS Shiratorizawa Ga...   \n",
       "6     11061                             Hunter x Hunter (2011)   \n",
       "7       820                               Ginga Eiyuu Densetsu   \n",
       "8     15335  Gintama Movie: Kanketsu-hen - Yorozuya yo Eien...   \n",
       "9     15417                           Gintama&#039;: Enchousen   \n",
       "\n",
       "                                               genre   type  episodes  rating  \\\n",
       "0               Drama, Romance, School, Supernatural  Movie         1    9.37   \n",
       "1  Action, Adventure, Drama, Fantasy, Magic, Mili...     TV        64    9.26   \n",
       "2  Action, Comedy, Historical, Parody, Samurai, S...     TV        51    9.25   \n",
       "3                                   Sci-Fi, Thriller     TV        24    9.17   \n",
       "4  Action, Comedy, Historical, Parody, Samurai, S...     TV        51    9.16   \n",
       "5             Comedy, Drama, School, Shounen, Sports     TV        10    9.15   \n",
       "6            Action, Adventure, Shounen, Super Power     TV       148    9.13   \n",
       "7                     Drama, Military, Sci-Fi, Space    OVA       110    9.11   \n",
       "8  Action, Comedy, Historical, Parody, Samurai, S...  Movie         1    9.10   \n",
       "9  Action, Comedy, Historical, Parody, Samurai, S...     TV        13    9.11   \n",
       "\n",
       "   members  Action  Adventure  Cars  ...  Slice of Life  Space  Sports  \\\n",
       "0   200630       0          0     0  ...              0      0       0   \n",
       "1   793665       1          1     0  ...              0      0       0   \n",
       "2   114262       1          0     0  ...              0      0       0   \n",
       "3   673572       0          0     0  ...              0      0       0   \n",
       "4   151266       1          0     0  ...              0      0       0   \n",
       "5    93351       0          0     0  ...              0      0       1   \n",
       "6   425855       1          1     0  ...              0      0       0   \n",
       "7    80679       0          0     0  ...              0      1       0   \n",
       "8    72534       1          0     0  ...              0      0       0   \n",
       "9    81109       1          0     0  ...              0      0       0   \n",
       "\n",
       "   Super Power  Supernatural  Thriller  Unknown  Vampire  Yaoi  Yuri  \n",
       "0            0             1         0        0        0     0     0  \n",
       "1            0             0         0        0        0     0     0  \n",
       "2            0             0         0        0        0     0     0  \n",
       "3            0             0         1        0        0     0     0  \n",
       "4            0             0         0        0        0     0     0  \n",
       "5            0             0         0        0        0     0     0  \n",
       "6            1             0         0        0        0     0     0  \n",
       "7            0             0         0        0        0     0     0  \n",
       "8            0             0         0        0        0     0     0  \n",
       "9            0             0         0        0        0     0     0  \n",
       "\n",
       "[10 rows x 51 columns]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "anime.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rec_cols = np.argsort(group_anime)[::-1]\n",
    "rec_cols = rec_cols[~np.isin(rec_cols, watched_cols)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "r_user = user_item_matrix.index.get_loc(user.values[0])\n",
    "groups = w1.argmax(axis=1)\n",
    "user_group = groups[r_user]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "group_anime = h1[user_group, :]\n",
    "print(len(group_anime), len(user_item_matrix.columns))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cols = user_item_matrix.loc[user].dropna(axis=1).columns\n",
    "# [cols.get_loc(col) for col in cols]\n",
    "[user_item_matrix.columns.get_loc(col) for col in cols]\n",
    "\n",
    "a,b = [2,4,6,8],[3,4,5] \n",
    "a[~np.isin(a,b)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "user"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "r_user = user_item_matrix.index.get_loc(user.values[0])\n",
    "watched_anime = user_item_matrix.loc[user].dropna(axis=1).columns\n",
    "watched_cols = np.array([user_item_matrix.columns.get_loc(col) for col in watched_anime])\n",
    "\n",
    "rec_cols = np.argsort(group_anime)[::-1]\n",
    "rec_cols = rec_cols[~np.isin(rec_cols, watched_cols)]\n",
    "\n",
    "rec_anime_id = user_item_matrix.iloc[r_user, rec_cols].index\n",
    "rec_weight = group_anime[rec_cols]# user_item_matrix.loc[user, cols[1]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rec_anime_id[:6], rec_weight[:6] / np.sum(rec_weight[:6])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "anime[anime['anime_id'].isin(rec_anime_id[:6])].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(group_anime)\n",
    "rec = group_anime[~np.isin(group_anime, user_item_matrix.loc[user].dropna(axis=1).columns)]\n",
    "print(rec)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rec = group_anime[np.argsort(group_anime)[::-1]]\n",
    "rec_anime[~np.isin(user_item_matrix.loc[user].dropna(axis=1).columns)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_item_matrix.loc[user].dropna(axis=1).columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "anime[anime['genre'].str.contains('School')].head()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}