ML.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fdf55623",
   "metadata": {},
   "source": [
    "# Project in Machine Learning\n",
    "Classification of objects - Pictures of Cats, Dogs and Wild Animals"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this project, we use a training set of 14630 pictures (512x512, jpg) of cats, dogs and wild animals in order to create a model that can recognize the animals in new images. Then, we test this model in a validation set of 1500 pictures."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Source of the dataset: https://www.kaggle.com/datasets/andrewmvd/animal-faces/ \\\n",
    "Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition \\\n",
    "Yunjey Choi and Youngjung Uh and Jaejun Yoo and Jung-Woo Ha, 2020 \\\n",
    "StarGAN v2: Diverse Image Synthesis for Multiple Domains"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23ebffae",
   "metadata": {},
   "source": [
    "### 1. Principal Component Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "738cf821",
   "metadata": {},
   "source": [
    "Ξεκινήσαμε συλλέγοντας όλα τα δεδομένα και μετατρέποντάς τα σε ασπρόμαυρα."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2b94e563",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import cv2\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "\n",
    "# Start timer\n",
    "start_time = time.time()\n",
    "print(start_time)\n",
    "\n",
    "# Load images and convert to grayscale\n",
    "def load_images(folder_path, size=(64, 64)):\n",
    "    images = []\n",
    "    for filename in os.listdir(folder_path):\n",
    "        img = cv2.imread(os.path.join(folder_path, filename))\n",
    "        if img is not None:\n",
    "            img = cv2.resize(img, size)  # Resize image\n",
    "            images.append(img)\n",
    "    return images\n",
    "\n",
    "# Function to flatten images to vectors\n",
    "def flatten_images(images):\n",
    "    flattened_images = []\n",
    "    for img in images:\n",
    "        flattened_images.append(img.flatten())\n",
    "    return np.array(flattened_images)\n",
    "\n",
    "# Load the dataset of images (cats, dogs, wild animals)\n",
    "train_folder_path = r\"/afhq/train/cat\"\n",
    "images = load_images(train_folder_path)\n",
    "\n",
    "# Convert images to grayscale\n",
    "grayscale_images = [cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) for img in images]\n",
    "\n",
    "# Flatten and convert images to numpy array (vectors)\n",
    "flattened_resized_images = flatten_images(grayscale_images)\n",
    "\n",
    "# Convert images to float64 data type\n",
    "flattened_resized_images = flattened_resized_images.astype('float64')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba8c2c36",
   "metadata": {},
   "source": [
    "Έπειτα, ορίσαμε την κλάση PCA, η οποία περιλαμβάνει μεθόδους για τον υπολογισμό της ιδιοαποσύνθεσης του πίνακα συμμεταβλητότητας και την έξοδο του PCA (ο τελικός πίνακας y). \\\n",
    "Η υπερπαράμετρος k είναι η self.n_components."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "847ddc35",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "# Implement PCA\n",
    "class PCA:\n",
    "    def __init__(self, n_components):\n",
    "        self.n_components = n_components\n",
    "        self.components = None #The Q Matrix\n",
    "        self.mean = None\n",
    "\n",
    "\n",
    "    def fit(self, X):\n",
    "        # Convert to float for calculations\n",
    "        X = X.astype('float64')\n",
    "\n",
    "        # Mean centering\n",
    "        self.mean = np.mean(X, axis=0)\n",
    "        X -= self.mean\n",
    "\n",
    "        # Calculate covariance matrix\n",
    "        cov_matrix = np.cov(X.T)\n",
    "\n",
    "        # Eigen decomposition\n",
    "        eigenvalues, eigenvectors = np.linalg.eig(cov_matrix)\n",
    "\n",
    "        # Sort eigenvalues and eigenvectors\n",
    "        eigenvectors = eigenvectors.T\n",
    "        idxs = np.argsort(eigenvalues)[::-1]\n",
    "        eigenvalues = eigenvalues[idxs]\n",
    "        eigenvectors = eigenvectors[idxs]\n",
    "        inv_2_eigenvalues = np.sqrt(np.linalg.inv(np.diag(eigenvalues)))\n",
    "\n",
    "        # Store first n_components eigenvectors\n",
    "        self.components = inv_2_eigenvalues @ eigenvectors.T [:self.n_components]\n",
    "\n",
    "    def transform(self, X):\n",
    "        # Mean centering\n",
    "        X -= self.mean\n",
    "\n",
    "        # Project data onto the components\n",
    "        return np.dot(X, self.components.T)\n",
    "\n",
    "\n",
    "    def fit_transform(self, X):\n",
    "        self.fit(X)\n",
    "        return self.transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71db765f",
   "metadata": {},
   "source": [
    "Τέλος, εφαρμόζουμε τις μεθόδους της κλάσης στις εικόνες και τυπώνουμε τα αποτελέσματα."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c26509d5",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "# The number of components\n",
    "n_components = 100 \n",
    "\n",
    "# Apply PCA to images\n",
    "pca = PCA(n_components=n_components)\n",
    "transformed_images = pca.fit_transform(flattened_resized_images)\n",
    "print(\"Applied PCA to images\")\n",
    "\n",
    "# Reconstruct images using the transformed data\n",
    "reconstructed_images = np.dot(transformed_images, pca.components) + pca.mean\n",
    "print(\"Reconstructed images using the transformed data\")\n",
    "\n",
    "# Reshape reconstructed images to their original shapes\n",
    "reconstructed_images = reconstructed_images.reshape(len(images), *grayscale_images[0].shape)\n",
    "print(\"Reshaped reconstructed images to their original shapes\")\n",
    "\n",
    "# Ensure reconstructed images are of appropriate data type (e.g., convert to uint8)\n",
    "reconstructed_images = np.real(reconstructed_images).astype('uint8')\n",
    "print(\"Ensure reconstructed images are of appropriate data type (e.g., convert to uint8)\")\n",
    "\n",
    "# Plot original and reconstructed images\n",
    "fig, axes = plt.subplots(2, 5, figsize=(10, 6))\n",
    "for i in range(5):\n",
    "    axes[0, i].imshow(grayscale_images[i], cmap='gray')\n",
    "    axes[0, i].axis('off')\n",
    "    axes[0, i].set_title('Original')\n",
    "\n",
    "    axes[1, i].imshow(reconstructed_images[i], cmap='gray')\n",
    "    axes[1, i].axis('off')\n",
    "    axes[1, i].set_title('Reconstructed')\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4471750",
   "metadata": {},
   "source": [
    "### 2. Least Squares Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3aab7521",
   "metadata": {},
   "source": [
    "Ξεκινάμε ξανά φορτώνοντας τις απαραίτητες βιβλιοθήκες και το dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a4a8993c",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import cv2\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "import random\n",
    "\n",
    "train_folder_path_cats = r\"/afhq/train/cat\"\n",
    "train_folder_path_dogs = r\"/afhq/train/dog\"\n",
    "train_folder_path_wild = r\"/afhq/train/wild\"\n",
    "images = load_images(train_folder_path_cats, 0)\n",
    "images.extend(load_images(train_folder_path_dogs), 1)\n",
    "images.extend(load_images(train_folder_path_wild), 2)\n",
    "\n",
    "#---\n",
    "\n",
    "def load_images(folder_path, class_name = None, size=(64, 64)):\n",
    "    images = []\n",
    "    image_class = []\n",
    "    for filename in os.listdir(folder_path):\n",
    "        img = cv2.imread(os.path.join(folder_path, filename))\n",
    "        if img is not None:\n",
    "            img = cv2.resize(img, size)  # Resize image\n",
    "            images.append(img)\n",
    "            image_class.append(class_name)\n",
    "    return images\n",
    "\n",
    "#Edit images so the become 512x512-dimensional\n",
    "images = [cv2.resize(img, (32, 32)) for img in images]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0339780d",
   "metadata": {},
   "source": [
    "Ορίσαμε την κλάση LeastSquaresModel για την διενέργεια του αλγορίθμου. \\\n",
    "Η μέθοδος calculate παράγει το αποτέλεσμα της w*x + b. \\\n",
    "Η μέθοδος error παράγει το σφάλμα μεταξύ των στόχων και των παραχθέντων αποτελεσμάτων \\\n",
    "Η μέθοδος fix υπολογίζει τα νέα weights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43ad47c5",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "class LeastSquaresModel:\n",
    "    def __init__(self):\n",
    "        self.weights = [x for x in random.sample(range(0, 1024), 1024)]\n",
    "        self.bias = 1\n",
    "    def calculate(self, x): #Function y = w*x + b\n",
    "        return sum([self.weights[i] * x[i] for i in range(len(self.weights))]) + self.bias\n",
    "    def error(self, x, t): #Function error -> t - y_predicted\n",
    "        return 1/2 * (t - self.calculate(x))**2\n",
    "    def fix(self, X, T): #Function to fix the error\n",
    "        self.weights = (np.array(X).T @ np.array(X))^-1 @ (np.array(X).T @ np.array(T))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4005af6b",
   "metadata": {},
   "source": [
    "Ορίσαμε και τις εξής υποστηρικτικές μεθόδους:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33ece979",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "def get_array_X(images):\n",
    "    X = []\n",
    "    for img in images:\n",
    "        img_array = np.array(img)\n",
    "        img_array = img_array.flatten()\n",
    "        X.append(img_array)\n",
    "    return X\n",
    "\n",
    "def get_array_T(image_class):\n",
    "    T = []\n",
    "    t = []\n",
    "    for ic in image_class:\n",
    "        if ic == 0:\n",
    "            t = [1, 0, 0]\n",
    "        if ic == 1:\n",
    "            t = [0, 1, 0]\n",
    "        if ic == 2:\n",
    "            t = [0, 0, 1]\n",
    "        T.append(t)\n",
    "    return T"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d142fb1",
   "metadata": {},
   "source": [
    "Τέλος, χρησιμοποιούμε την κλάση LeastSquaresModel για να δημιουργήσουμε ένα μοντέλο και να το εκπαιδεύσουμε."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14ac26f4",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "#Initializing the model\n",
    "model = LeastSquaresModel()\n",
    "\n",
    "#Train\n",
    "model.fix(get_array_X(images), get_array_T(image_class))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "600744ad",
   "metadata": {},
   "source": [
    "### 3. Logistic Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2e34722",
   "metadata": {},
   "source": [
    "Όμοια με πριν, φορτώνουμε αρχικά τα data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b1cf7167",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import cv2\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "import random\n",
    "\n",
    "#Loading images\n",
    "train_folder_path_cats = r\"/afhq/train/cat\"\n",
    "train_folder_path_dogs = r\"/afhq/train/dog\"\n",
    "train_folder_path_wild = r\"/afhq/train/wild\"\n",
    "images, targets = load_images(train_folder_path_cats, [1,0,0])\n",
    "res = load_images(train_folder_path_dogs, [0,1,0])\n",
    "images.extend(res[0])\n",
    "targets.extend(res[1])\n",
    "res = load_images(train_folder_path_wild, [0,0,1])\n",
    "images.extend(res[0])\n",
    "targets.extend(res[1])\n",
    "\n",
    "def load_images(folder_path, class_name = None, size=(64, 64)):\n",
    "    images = []\n",
    "    image_class = []\n",
    "    for filename in os.listdir(folder_path):\n",
    "        img = cv2.imread(os.path.join(folder_path, filename))\n",
    "        if img is not None:\n",
    "            img = cv2.resize(img, size)  # Resize image\n",
    "            images.append(img)\n",
    "            image_class.append(class_name)\n",
    "    return images, image_class #targets\n",
    "\n",
    "#Reducing image size\n",
    "images = [cv2.resize(img, (32, 32)) for img in images]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ca4807d",
   "metadata": {},
   "source": [
    "Στην συνέχεια, παράγουμε softmax πιθανότητες για κάθε εικόνα."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf7bc0dd",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "#Initializing b vectors for each class\n",
    "b_class_cats = np.randints(32*32)\n",
    "b_class_dogs = np.randints(32*32)\n",
    "b_class_wild = np.randints(32*32)\n",
    "\n",
    "def getClass_Probabilities(image, b_class_cats, b_class_dogs, b_class_wild):\n",
    "    e_power_cats = Math.pow(2.71, np.dot(image, b_class_cats))\n",
    "    e_power_dogs = Math.pow(2.71, np.dot(image, b_class_dogs))\n",
    "    e_power_wild = Math.pow(2.71, np.dot(image, b_class_wild))\n",
    "    sum = e_power_cats + e_power_dogs + e_power_wild\n",
    "    pr_cats = e_power_cats / sum\n",
    "    pr_dogs = e_power_dogs / sum\n",
    "    pr_wild = e_power_wild / sum\n",
    "    if pr_cats > pr_dogs and pr_cats > pr_wild:\n",
    "        return np.vectors.array([1, 0, 0]), np.vectors.array([pr_cats, pr_dogs, pr_wild]) #Cat\n",
    "    elif pr_dogs > pr_cats and pr_dogs > pr_wild:\n",
    "        return np.vectors.array([0, 1, 0]), np.vectors.array([pr_cats, pr_dogs, pr_wild]) #Dog\n",
    "    else:\n",
    "        return np.vectors.array([0, 0, 1]), np.vectors.array([pr_cats, pr_dogs, pr_wild]) #Wild"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "735e7124",
   "metadata": {},
   "source": [
    "Τα b vectors διορθώνονται καθώς εκπαιδεύεται το μοντέλο μέσω του cross entropy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5aab8503",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "def error(images, targets, b_class_cats, b_class_dogs, b_class_wild):\n",
    "    cross_entropy = 0\n",
    "    for i in range(len(images)):\n",
    "        image = images[i]\n",
    "        target = targets[i]\n",
    "        pr_res = getClass_Probabilities(image, b_class_cats, b_class_dogs, b_class_wild)[1]\n",
    "        for j in range(len(pr_res)):\n",
    "            cross_entropy -= target[j] * Math.log(pr_res[j])\n",
    "    return cross_entropy\n",
    "\n",
    "#Applying Logistic Regression\n",
    "for i in range(30):\n",
    "    err = error(images, targets, b_class_cats, b_class_dogs, b_class_wild)\n",
    "    alpha = 0.01\n",
    "    #Applying regression with cross-entropy\n",
    "    b_class_cats = b_class_cats + alpha * err\n",
    "    b_class_dogs = b_class_dogs + alpha * err\n",
    "    b_class_wild = b_class_wild + alpha * err"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a35c992f",
   "metadata": {},
   "source": [
    "### 4. K Nearest Neighbors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "79e5a3d2",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import cv2\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "import random\n",
    "\n",
    "images = []\n",
    "labels = []\n",
    "targets = []\n",
    "\n",
    "#--\n",
    "\n",
    "train_folder_path_cats = r\"/afhq/train/cat\"\n",
    "train_folder_path_dogs = r\"/afhq/train/dog\"\n",
    "train_folder_path_wild = r\"/afhq/train/wild\"\n",
    "images, targets = load_images(train_folder_path_cats, [1,0,0])\n",
    "res = load_images(train_folder_path_dogs, [0,1,0])\n",
    "images.extend(res[0])\n",
    "targets.extend(res[1])\n",
    "res = load_images(train_folder_path_wild, [0,0,1])\n",
    "images.extend(res[0])\n",
    "targets.extend(res[1])\n",
    "\n",
    "def load_images(folder_path, class_name = None, size=(64, 64)):\n",
    "    images = []\n",
    "    image_class = []\n",
    "    for filename in os.listdir(folder_path):\n",
    "        img = cv2.imread(os.path.join(folder_path, filename))\n",
    "        if img is not None:\n",
    "            img = cv2.resize(img, size)  # Resize image\n",
    "            images.append(img)\n",
    "            image_class.append(class_name)\n",
    "    return images, image_class #targets\n",
    "\n",
    "#Reducing image size\n",
    "images = [cv2.resize(img, (32, 32)) for img in images]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4113bd3",
   "metadata": {},
   "source": [
    "Το dataset φορτώνεται. Μέσω της μεθόδου find_class ταξινομείται η εικόνα αναλόγως των k κοντινότερων γειτόνων της. Η error επιστρέφει το τετραγωνικό σφάλμα της εκτίμησης. \\\n",
    "Οι πρώτες 8 εικόνες χρησιμοποιούνται για την εκπαίδευση του μοντέλου."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "afd3020e",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "def find_K_Nearest_Neighbors(X, T, k): # X = data, T = new image to be added to data\n",
    "    distances = []\n",
    "    for i in range(len(X)):\n",
    "        distances.append(np.linalg.norm(X[i] - T))\n",
    "    distances = np.array(distances)\n",
    "    indices = distances.argsort()[:k]\n",
    "    return indices\n",
    "\n",
    "def find_class(X, T, k):\n",
    "    indices = find_K_Nearest_Neighbors(X, T, k)\n",
    "    class_counting_list = [0, 0, 0]\n",
    "    for indice in indices:\n",
    "        class_counting_list[labels[indice]-1] += 1\n",
    "    return np.argmax(class_counting_list)\n",
    "\n",
    "def add_Image_To_Dataset(data, labels, image):\n",
    "    data.append(image)\n",
    "    labels.append(find_class(data, image, 3))\n",
    "    return data, labels\n",
    "    \n",
    "def error(labels, targets):\n",
    "    return 1/2 * (np.array(labels) - np.array(targets))**2\n",
    "\n",
    "def reset():\n",
    "    x = []\n",
    "    return x\n",
    "\n",
    "#--\n",
    "\n",
    "for i in range(1,11):\n",
    "    k = i\n",
    "    print(\"K = \", k)\n",
    "    data = reset()\n",
    "    data = images[:8]\n",
    "    labels = reset()\n",
    "    labels = targets[:8]\n",
    "    for i in range(8, len(images)):\n",
    "        data, labels = add_Image_To_Dataset(data, labels, images[i])\n",
    "    print(\"Error: \", error(labels, targets))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77cc4bad",
   "metadata": {},
   "source": [
    "### 5. Naive Bayes "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c7d1510",
   "metadata": {},
   "source": [
    "Αρχικά, φορτώνεται το dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6949938e",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import cv2\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "import random\n",
    "\n",
    "#Loading images\n",
    "train_folder_path_cats = r\"/afhq/train/cat\"\n",
    "train_folder_path_dogs = r\"/afhq/train/dog\"\n",
    "train_folder_path_wild = r\"/afhq/train/wild\"\n",
    "images, targets = load_images(train_folder_path_cats, [1,0,0])\n",
    "res = load_images(train_folder_path_dogs, [0,1,0])\n",
    "images.extend(res[0])\n",
    "targets.extend(res[1])\n",
    "res = load_images(train_folder_path_wild, [0,0,1])\n",
    "images.extend(res[0])\n",
    "targets.extend(res[1])\n",
    "\n",
    "def load_images(folder_path, class_name = None, size=(64, 64)):\n",
    "    images = []\n",
    "    image_class = []\n",
    "    for filename in os.listdir(folder_path):\n",
    "        img = cv2.imread(os.path.join(folder_path, filename))\n",
    "        if img is not None:\n",
    "            img = cv2.resize(img, size)  # Resize image\n",
    "            images.append(img)\n",
    "            image_class.append(class_name)\n",
    "    return images, image_class #targets\n",
    "\n",
    "#Reducing image size\n",
    "images = [cv2.resize(img, (8, 8)) for img in images]\n",
    "\n",
    "#Convert to grayscale\n",
    "images = [cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) for img in images]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da57f789",
   "metadata": {},
   "source": [
    "Στην συνέχεια, βάσει αυτού, υπολογίζονται οι απαραίτητες πιθανότητες που χρειάζονται για την ταξινόμηση των εικόνων."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "daf113ee",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "#Counting colors + calculating probabilities\n",
    "color_counter = np.zeros(256)\n",
    "color_probabilities_sum = [[0, 0, 0] for _ in range(256)]\n",
    "\n",
    "for k in range(len(images)):\n",
    "    img = images[k]\n",
    "    for i in range(32):\n",
    "        for j in range(32):\n",
    "            color_counter[img[i,j]] += 1\n",
    "            if targets[k] == [1,0,0]:\n",
    "                color_probabilities_sum[img[i,j]][0] += 1\n",
    "            if targets[k] == [0,1,0]:\n",
    "                color_probabilities_sum[img[i,j]][1] += 1\n",
    "            if targets[k] == [0,0,1]:\n",
    "                color_probabilities_sum[img[i,j]][2] += 1\n",
    "\n",
    "color_probabilities = [[0,0,0] for _ in range(256)] # Pr[Class_i = class_j | ColorX = colorx]\n",
    "for i in range(256):\n",
    "    if color_counter[i]!= 0:\n",
    "        color_probabilities[i] = [color_probabilities_sum[i][0]/color_counter[i], color_probabilities_sum[i][1]/color_counter[i], color_probabilities_sum[i][2]/color_counter[i]]\n",
    "\n",
    "class_probabilities = [[0,0,0] for _ in range(256)] # Pr[ColorX = colorx | Class_i = class_j]\n",
    "class_total_count = [0, 0, 0] # [x_i = class_i]\n",
    "for i in range(256):\n",
    "    summ = class_probabilities_sum[i][0] + class_probabilities_sum[i][1] + class_probabilities_sum[i][2]\n",
    "    for j in range(3):\n",
    "        class_probabilities[i][j] += color_probabilities_sum[i][j]\n",
    "        class_total_count[j] += color_probabilities_sum[i][j]\n",
    "        if summ!= 0:\n",
    "            class_probabilities[i][j] /= summ\n",
    "        #...\n",
    "\n",
    "#Calculating prior probabilities Pr[Class_i = class_i]\n",
    "prior_probabilities = [0,0,0]\n",
    "for i in range(3):\n",
    "    prior_probabilities[i] = class_total_count[i] / len(images)\n",
    "\n",
    "def product_calculator(image, k):  # γινόμενο όλων των P(xi|Ck)\n",
    "    product = 1\n",
    "    for i in range(256):\n",
    "        product *= class_probabilities[i][k]\n",
    "    return product"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98e54341",
   "metadata": {},
   "source": [
    "Τέλος, εφαρμόζεται ο αλγόριθμος ταξινόμησης."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5ee2861a",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "#Naive-Bayes Classifier\n",
    "def naive_bayes_classifier(image):\n",
    "    maxx = -1\n",
    "    maxx_class = [0,0,0]\n",
    "    for i in range(3): # For each class\n",
    "        if prior_probabilities[i]*product_calculator(image, i) > maxx:\n",
    "            maxx = prior_probabilities[i]*product_calculator(image, i)\n",
    "            maxx_class = [0,0,0]\n",
    "            maxx_class[i] = 1\n",
    "    return maxx_class\n",
    "\n",
    "for image in images:\n",
    "    print(naive_bayes_classifier(image))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d74f9075",
   "metadata": {},
   "source": [
    "### 6. Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1c196ac",
   "metadata": {},
   "source": [
    "Στην αρχή φορτώνεται το dataset και οι απαραίτητες βιβλιοθήκες για την ανάπτυξη του νευρωνικού δικτύου."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d05e4adc",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import cv2\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "import random\n",
    "import torch\n",
    "import torchvision.models as models\n",
    "\n",
    "#Loading images\n",
    "train_folder_path_cats = r\"/afhq/train/cat\"\n",
    "train_folder_path_dogs = r\"/afhq/train/dog\"\n",
    "train_folder_path_wild = r\"/afhq/train/wild\"\n",
    "images, targets = load_images(train_folder_path_cats, [1,0,0])\n",
    "res = load_images(train_folder_path_dogs, [0,1,0])\n",
    "images.extend(res[0])\n",
    "targets.extend(res[1])\n",
    "res = load_images(train_folder_path_wild, [0,0,1])\n",
    "images.extend(res[0])\n",
    "targets.extend(res[1])\n",
    "\n",
    "def load_images(folder_path, class_name = None, size=(64, 64)):\n",
    "    images = []\n",
    "    image_class = []\n",
    "    for filename in os.listdir(folder_path):\n",
    "        img = cv2.imread(os.path.join(folder_path, filename))\n",
    "        if img is not None:\n",
    "            img = cv2.resize(img, size)  # Resize image\n",
    "            images.append(img)\n",
    "            image_class.append(class_name)\n",
    "    return images, image_class #targets\n",
    "\n",
    "#Reducing image size\n",
    "images = [cv2.resize(img, (8, 8)) for img in images]\n",
    "\n",
    "#Convert to grayscale\n",
    "images = [cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) for img in images]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6266e87b",
   "metadata": {},
   "source": [
    "Έπειτα ορίζουμε και εκπαιδεύουμε ένα νευρωνικό δίκτυο μέσω της βιβλιοθήκης Torch και τυπώνουμε τα αποτελέσματα ανά εικόνα, παράλληλα συγκρίνοντας με το επιθυμητό αποτέλεσμα."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c2f73319",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "#Building the initial neural network\n",
    "model = models.resnet18(pretrained=True)\n",
    "\n",
    "#Training the network\n",
    "model.fc = torch.nn.Linear(512, 3)\n",
    "epochs = 3\n",
    "criterion = torch.nn.MSELoss()\n",
    "optimizer = torch.optim.SGD(model.parameters(), lr=0.001, momentum=0.9)\n",
    "compiled_model = torch.compile(model, criterion, optimizer, epochs)\n",
    "for i in range(epochs):\n",
    "    compiled_model.train(images, targets)\n",
    "\n",
    "#Results\n",
    "print(\"Results:\")\n",
    "for i in range(len(images)):\n",
    "    img = images[i]\n",
    "    img = torch.tensor(img)\n",
    "    pred = compiled_model.predict(img)\n",
    "    pred = pred.detach().numpy()\n",
    "    pred = np.argmax(pred)\n",
    "    print(\"Image: \", i)\n",
    "    print(\"Prediction: \", pred)\n",
    "    print(\"Target: \", targets[i])\n",
    "    print(\"--------------------------------\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08970457",
   "metadata": {},
   "source": [
    "### 7. K Means"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b8c64a5",
   "metadata": {},
   "source": [
    "Αρχικά φορτώνεται το dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "62e37548",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import cv2\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "import random\n",
    "\n",
    "#Loading images\n",
    "train_folder_path_cats = r\"/afhq/train/cat\"\n",
    "train_folder_path_dogs = r\"/afhq/train/dog\"\n",
    "train_folder_path_wild = r\"/afhq/train/wild\"\n",
    "images, targets = load_images(train_folder_path_cats, [1,0,0])\n",
    "res = load_images(train_folder_path_dogs, [0,1,0])\n",
    "images.extend(res[0])\n",
    "targets.extend(res[1])\n",
    "res = load_images(train_folder_path_wild, [0,0,1])\n",
    "images.extend(res[0])\n",
    "targets.extend(res[1])\n",
    "\n",
    "def load_images(folder_path, class_name = None, size=(64, 64)):\n",
    "    images = []\n",
    "    image_class = []\n",
    "    for filename in os.listdir(folder_path):\n",
    "        img = cv2.imread(os.path.join(folder_path, filename))\n",
    "        if img is not None:\n",
    "            img = cv2.resize(img, size)  # Resize image\n",
    "            images.append(img)\n",
    "            image_class.append(class_name)\n",
    "    return images, image_class #targets\n",
    "\n",
    "#Reducing image size\n",
    "images = [cv2.resize(img, (8, 8)) for img in images]\n",
    "\n",
    "#Convert to grayscale\n",
    "images = [cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) for img in images]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a571df1",
   "metadata": {},
   "source": [
    "Έπειτα ορίζουμε την μέθοδο kmeans η οποία σε 2 φάσεις, πρώτα πραγματοποιεί την εκπαίδευση και μετά την εφαρμογή του αλγορίθμου."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7a327a8",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "#Number of means\n",
    "k = 3\n",
    "\n",
    "def kmeans(images, targets, k):\n",
    "    #Initialize means with random numbers in [0,256]\n",
    "    means = np.random.randint(0, 256, (k, 8, 8))\n",
    "    neighbors = np.zeros(len(images))\n",
    "\n",
    "    #Train\n",
    "    for epochs in range(10):\n",
    "        for i in range(len(images)):\n",
    "            img = images[i]\n",
    "            #Calculate distances\n",
    "            distances = np.zeros(k)\n",
    "            for j in range(k):\n",
    "                distances[j] = np.linalg.norm(img - means[j])\n",
    "            neighbors[i] = argmin(distances)\n",
    "        #Calculate new means\n",
    "        for i in range(k):\n",
    "            means[i] = np.mean(images[neighbors == i], axis=0)\n",
    "\n",
    "    #Test\n",
    "    for i in range(len(images)):\n",
    "        img = images[i]\n",
    "        #Calculate distances\n",
    "        distances = np.zeros(k)\n",
    "        for j in range(k):\n",
    "            distances[j] = np.linalg.norm(img - means[j])\n",
    "        neighbors[i] = argmin(distances)\n",
    "        #Print results\n",
    "        if (targets[neighbors[i]] == targets[i]):\n",
    "            print(\"Success\")\n",
    "        else:\n",
    "            print(\"Error\")\n",
    "\n",
    "def argmin(array):\n",
    "    return np.argmin(array)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1af55ac1",
   "metadata": {},
   "source": [
    "Τέλος, εφαρμόζουμε τη μέθοδο στο dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad965f3d",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "outputs": [],
   "source": [
    "kmeans(images, targets, k)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}