{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "98848cc1",
   "metadata": {},
   "source": [
    "# Lesson 33 Object Detection & More\n",
    "NOTE: This Lesson assumes python Python 3.12.12 so that you can use `label-studio`.\n",
    "\n",
    "**However**, all YOLO Models work just fine on the latest 3.14 version of python. And for using Roboflow to annotate images. I recommend python version : 3.14"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e507302",
   "metadata": {},
   "source": [
    "![CLDvS](https://gperdrizet.github.io/FSA_devops/assets/lesson_33_images/CLDvS.png)\n",
    "\n",
    "- **Classification** *(Is there a cat or no cat?)*\n",
    "    - You could perfrom multi-class or multi-label too!\n",
    "        - **MC**: [0.90: \"Cat\", 0.04: \"Dog\", 0.01: \"Rat\"]\n",
    "        - **ML**: {0.99 : 'Animal', 0.7 : 'Cat'}\n",
    "---\n",
    "- **Localization** *(Where is the Cat?)*\n",
    "    - You could even find multiple Cats!! \n",
    "        - Cat_1 : [x1, x2, y1, y2]\n",
    "        - Cat_2: [x3, x4, y3, y4]...\n",
    "---\n",
    "- **Object Detection** *(Where are the Cats? Oh, yeah I can locate dogs too!)*\n",
    "    - Obj_1: \n",
    "        - Localization: [x1, x2, y1, y2]\n",
    "        - Classification: [0.90: \"Cat\", 0.04: \"Dog\", 0.01: \"Rat\"] : \"Cat\"\n",
    "    - Obj_2: \n",
    "        - Localization: [x3, x4, y3, y4]\n",
    "        - Classification: [0.01: \"Cat\", 0.98: \"Dog\", 0.01: \"Rat\"] : \"Dog\"\n",
    "    - Obj_3: \n",
    "        - Localization: [x5, x6, y5, y6]\n",
    "        - Classification: [0.00: \"Cat\", 0.95: \"Dog\", 0.05: \"Rat\"] : \"Dog\"\n",
    "---\n",
    "- **Image Segmentaion** *(I'm creating a scrap-book of cute animals!)*\n",
    "    - =^..^= \n",
    "        - Classification: [0.90: \"Cat\", 0.04: \"Dog\", 0.01: \"Rat\"] : \"Cat\" \n",
    "    - ˁ(OᴥO)ˀ\n",
    "        - Classification: [0.01: \"Cat\", 0.98: \"Dog\", 0.01: \"Rat\"] : \"Dog\"\n",
    "    - ) _ _ __/°°¬      \n",
    "        - Classification: [0.00: \"Cat\", 0.95: \"Dog\", 0.05: \"Rat\"] : \"Dog\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2d756ec",
   "metadata": {},
   "source": [
    "## Classification and Localaization"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9ac6e71",
   "metadata": {},
   "source": [
    "### Classification - Qualities \n",
    "#### What in this Image\n",
    "Captcha: Select all the images with Muffins.\n",
    "\n",
    "![Muffins](https://gperdrizet.github.io/FSA_devops/assets/lesson_33_images/select_all_muffings.png)\n",
    "\n",
    "Dataset Example:\n",
    "https://universe.roboflow.com/jacob-solawetz/flowers_classification"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6eda374",
   "metadata": {},
   "source": [
    "- Think back to our Custom CNN to using the CFAR10 dataset\n",
    "\n",
    "        - Classifies an image on which object it most likely contained\n",
    "        - Only predicts : Are any ___ Present?\n",
    "        - Predictions contain NO spacial coordinates\n",
    "       \n",
    "- Models could be trained to perform  multi-label predictions\n",
    "\n",
    "       - ['car', 'red'] --> 'red car'\n",
    "       - ['lizard', 'green'] --> 'green lizard'\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4070489",
   "metadata": {},
   "source": [
    "### Localization - Quantities\n",
    "#### Where in this Image?\n",
    "Captcha: Select all the images which contain traffic lights.\n",
    "\n",
    "![Overthinking](https://gperdrizet.github.io/FSA_devops/assets/lesson_33_images/localize_traffic_lights.png)\n",
    "\n",
    "- Could imagine creating a dataset of one object type with object coordinates\n",
    "    - Could have mutliple of the SAME object within an image.\n",
    "            \n",
    "            - Ball_1, Ball_2, ... , Ball_N-1, Ball_N\n",
    "\n",
    "        - Training a model to predict the coordinates of the object(s) within an image.\n",
    "        - Compare predictions to 'true label' coordinates.\n",
    "- Limitations:\n",
    "    - Only predicts : Where are ___?\n",
    "    - Predictions contain NO\n",
    "- Advantages:\n",
    "    - Model will find all the locations of an object within an image.\n",
    "    - If you add up how many locations are found, then the model implicitly learns to 'count'.\n",
    "\n",
    "            Object A : [x1, x2, y1, y2]\n",
    "            Object B : [x1, x2, y1, y2]  +\n",
    "            Object C : [x1, x2, y1, y2]  +\n",
    "            _______________________________\n",
    "            3 Object Instances in the Image\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49b8b3a9",
   "metadata": {},
   "source": [
    "## Detection, Segmentation, Pose Estimation, etc.. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7131fe71",
   "metadata": {},
   "source": [
    "### Object Detection\n",
    "- Detection:\n",
    "    Cutting out Rectangular Area that contains objects using a \"Guillotine\"\n",
    "    - Tries to Say: here is an object as quickly as possible\n",
    "    - Fast; High \"*Accuracy*\", with Low Area \"*Precision*\"\n",
    "\n",
    "#### Dataset used to train YOLO : https://cocodataset.org/#explore\n",
    "\n",
    "https://docs.ultralytics.com/tasks/detect/\n",
    "\n",
    "Dataset Example: \n",
    "https://universe.roboflow.com/bookspines/book-spines-fi8nq"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8108ab0e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from ultralytics import YOLO    \n",
    "import torch\n",
    "import gc # Garbage Collector\n",
    "\n",
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
    "print(f'Using device: {device}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dfd08cd1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load a Model\n",
    "model = YOLO(model='yolo26m.pt')    # Change the letter at the end [n, s, m, l x] for model with more parameters\n",
    "                                    # Gain 'better' metric performance at the cost of speed, bigger is slower\n",
    "                                    # Here we use the 'Medium' sized model: m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3703ac6e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Inference (Make Predictions)\n",
    "results = model(source=\"https://gperdrizet.github.io/FSA_devops/assets/lesson_33_images/select_all_muffings.png\", save=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49f6ba4d",
   "metadata": {},
   "outputs": [],
   "source": [
    "del model\n",
    "torch.cuda.empty_cache()\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "df2657d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Access the results\n",
    "for result in results:\n",
    "    xywh = result.boxes.xywh  # center-x, center-y, width, height\n",
    "    xywhn = result.boxes.xywhn  # normalized\n",
    "    xyxy = result.boxes.xyxy  # top-left-x, top-left-y, bottom-right-x, bottom-right-y\n",
    "    xyxyn = result.boxes.xyxyn  # normalized\n",
    "    names = [result.names[cls.item()] for cls in result.boxes.cls.int()]  # class name of each box\n",
    "    confs = result.boxes.conf  # confidence score of each box\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53492e64",
   "metadata": {},
   "source": [
    "### Objection Segmentation\n",
    "\n",
    "- Segmentaion:\n",
    "    Carefully cutting out Exact Objects using an \"Exacto Knife\"\n",
    "    - Tries to precisely locate an instance of an object within an image.\n",
    "    - Slower; High \"*Accuracy*\", with High Area \"*Precision*\"\n",
    "\n",
    "https://docs.ultralytics.com/tasks/segment/\n",
    "\n",
    "Dataset Example: https://universe.roboflow.com/atheer-algarni-gvico/car-parts-human-in-the-loop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9843baff",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load a Segmentaion Model\n",
    "seg_model = YOLO(model='yolo26m-seg.pt')    # Change the letter in the middle [n, s, m, l x] for model with more parameters\n",
    "                                            # Gain 'better' metric perfromance at the cost of speed, bigger is slower\n",
    "                                            # Here we use the 'Medium' sized model: m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f87d5482",
   "metadata": {},
   "outputs": [],
   "source": [
    "seg_results = seg_model(source = \"https://www.youtube.com/watch?v=W3zPhqBXaoM\", \n",
    "                        save = True, \n",
    "                        project = \"Cattle\")     # Save = True to get a visual of what the heck is going on with predictions\n",
    "                                                # Can easily view with mpv (or whatever method you use to view .avi file types)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e6fae650",
   "metadata": {},
   "outputs": [],
   "source": [
    "# seg_results = seg_model(source=0, \n",
    "#                         save=True, \n",
    "#                         project = \"Video Cam\")  # Save = True to get a visual of what the heck is going on with predictions\n",
    "#                                                 # Can easily view with mpv (or whatever method you use to view .avi file types)\n",
    "#                                                 # Source of 0 means use the webcam"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5afe7d23",
   "metadata": {},
   "outputs": [],
   "source": [
    "del seg_model\n",
    "torch.cuda.empty_cache()\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84383a2d",
   "metadata": {},
   "source": [
    "### Pose Estimation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0f67e46d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load Pose Model\n",
    "model = YOLO(model='yolo26m-pose.pt')   # Change the letter at the end [n, s, m, l x] for model with more parameters\n",
    "                                        # Gain 'better' metric perfromance at the cost of speed, bigger is slower\n",
    "                                        # Here we use the 'Medium' sized model: m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e267ae3",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.track(source=\"https://www.youtube.com/watch?v=YzcawvDGe4Y\", \n",
    "            save=True, \n",
    "            project=\"Busy Street\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3362c559",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.track(source=\"https://www.youtube.com/watch?v=2bYX8QcZ8mc\", \n",
    "            save=True, \n",
    "            project=\"Griddy\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4cdc2066",
   "metadata": {},
   "outputs": [],
   "source": [
    "del model\n",
    "torch.cuda.empty_cache()\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d412430",
   "metadata": {},
   "source": [
    "## Training your own YOLO Model - CIFAR-10\n",
    "### Classification is the easiest kind of dataset to get up and running\n",
    "Look into \n",
    "\n",
    "\n",
    "https://docs.ultralytics.com/datasets/\n",
    "\n",
    "https://docs.ultralytics.com/usage/callbacks/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "784b7312",
   "metadata": {},
   "source": [
    "Arbitrary keyword arguments for training configuration. Common options include:\n",
    "- data (str): Path to dataset configuration file.\n",
    "- epochs (int): Number of training epochs.\n",
    "- batch (int): Batch size for training.\n",
    "- imgsz (int): Input image size.\n",
    "- device (str): Device to run training on (e.g., 'cuda', 'cpu').\n",
    "- workers (int): Number of worker threads for data loading.\n",
    "- optimizer (str): Optimizer to use for training.\n",
    "- lr0 (float): Initial learning rate.\n",
    "- patience (int): Epochs to wait for no observable improvement for early stopping of training.\n",
    "- augmentations (list[Callable]): List of augmentation functions to apply during training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "09091a77",
   "metadata": {},
   "outputs": [],
   "source": [
    "from ultralytics import YOLO\n",
    "\n",
    "# Load a model\n",
    "model = YOLO(\"yolo26n-cls.pt\")  # load a pretrained model (recommended for training)\n",
    "\n",
    "# Train the model 1:33 with batch of 8192\n",
    "results = model.train(data=\"cifar10\", epochs=3, imgsz=32, batch=2048, device=device)\n",
    "\n",
    "# Export the model\n",
    "model.export(format=\"onnx\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34db91d5",
   "metadata": {},
   "outputs": [],
   "source": [
    "del model\n",
    "torch.cuda.empty_cache()\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4ae2468",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load a model\n",
    "model = YOLO(\"yolov8n-obb.pt\")  # load a pretrained model (recommended for training, but could modify)\n",
    "\n",
    "# Train the model 1:33 with batch of 8192\n",
    "# results = model.train(data=\"./datasets/Books/data.yaml\", epochs=3, device=device) # https://universe.roboflow.com/bookspines/book-spines-fi8nq/dataset/6/download  --> download zip file and point the data parameter to the .yaml file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cef55f9f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Export the model\n",
    "# model.export(format=\"onnx\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}