> For the complete documentation index, see [llms.txt](https://davids-tutorials.gitbook.io/robotics/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://davids-tutorials.gitbook.io/robotics/ignore-extra/computer-vision-tensorflow-lite.md).

# Computer Vision (TensorFlow Lite)

program that can find patterns or make decisions from a previously unseen dataset.

 Main steps

> 1 - Video/Roboflow: Gather images for dataset
>
> 2 - Colab: Train Custom Model
>
> 3 - Export Model
>
> 4 - Set up Pi
>
> 5 - Run on Pi

### 1 - Video/Roboflow: Gather images for dataset

| Logo                                                                                                                                                                                                                                                     | Description                                                                                                                                                                                                                           |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| <img src="https://439555438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FElvkNRN6rYA6sAuRlm5I%2Fuploads%2FwmxT6JmmV3WHUPyvgOUn%2Fimage.png?alt=media&amp;token=84e3dc49-607b-45f2-a85e-b5b474eedb73" alt="" data-size="original"> | We can film a video of the item we want to detect and then upload to Roboflow. On the Roboflow website, we can extract frames every X seconds to create a .jpg file each interval. Then we can begin labeling each image on Roboflow. |

> Roboflow is a website where you upload images, label them, and export a zip file (images & .xml files).&#x20;
>
> \
> Useful to share dataset and have 3 people label at the same time

### 1.1 - Unlabeled Images to Labeled Images&#x20;

| Unlabled                                                                                                                                                                                                                                                 | Labeled (Desired Outcome)                                                                                                                                                                                                                                |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| <img src="https://439555438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FElvkNRN6rYA6sAuRlm5I%2Fuploads%2FBGAcFzeGQqSAiOtpQVnE%2Fimage.png?alt=media&amp;token=791dda0e-298b-4a5f-9b4a-0fc3794513dd" alt="" data-size="original"> | <img src="https://439555438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FElvkNRN6rYA6sAuRlm5I%2Fuploads%2FLDpIXUBIM1OzRT2tQXh4%2Fimage.png?alt=media&amp;token=e37422f4-12f0-4e5f-99c0-5afe7cd0bc2f" alt="" data-size="original"> |

{% embed url="<https://www.youtube.com/watch?v=a3SBRtILjPI>" %}

### 2 - Colab: Train Custom Model

{% hint style="info" %}
&#x20;**WARNING**: This Colab began having issues in May 2023.&#x20;
{% endhint %}

<table><thead><tr><th width="365">Logo</th><th>Description</th></tr></thead><tbody><tr><td><img src="https://439555438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FElvkNRN6rYA6sAuRlm5I%2Fuploads%2FOclF7SI6nijqd1Juthoq%2Fimage.png?alt=media&amp;token=68f8210d-1f1b-404b-bf61-ffcdec7e78e3" alt="" data-size="original"></td><td><p>-The following website uses Python </p><p>-here we set up libraries </p><p>     - collections of pre-written code that you </p><p>        can use to perform specific tasks</p><p>-Prepare the dataset</p><p>-Train the model</p><p>-Evaluate the model performance with validation data</p><p>-Export the model as .tflite file </p></td></tr></tbody></table>

Open up this pre-made online coding website called [Google Colab](https://colab.research.google.com/github/khanhlvg/tflite_raspberry_pi/blob/main/object_detection/Train_custom_model_tutorial.ipynb)

### 3 - Export Model

We can export the custom model after training as a .tflite file which will be uploaded to the Raspberry Pi and then can be used for Object Detection. [Demo here](https://youtu.be/-ZyFYniGUsw?t=627)

### 4 - Set up Pi

| Original Tensorflow Video - 2 yrs ago                                                            | November 2023 - Recent Video                                |
| ------------------------------------------------------------------------------------------------ | ----------------------------------------------------------- |
| [YouTube Link](https://youtube.com/watch?v=mNjXEybFn98\&list=PLQY2H8rRoyvz_anznBg6y3VhuSMcpN9oe) | [YouTube Link](https://www.youtube.com/watch?v=ldvlAMcGWIk) |
| 0cyyUyl9smt2-hWdK8ivKExIY                                                                        | 0cyyUyl9smt2-5gal0Pt2blrl                                   |

| Colab Versions                        |
| ------------------------------------- |
| numpy 1.23                            |
| opencv-python-4.9.0.80                |
| tflite-support==0.4.2                 |
| pip install protobuf==3.20.\*         |
| <p>NOTE::<br>- Python 3.9.18</p>      |
| - pip install ipykernel               |
| - conda create -n myenv python=3.9 -y |
| - pip 23.3.1                          |
| - pycocotools-2.0.7                   |

| Pi Versions                     |
| ------------------------------- |
| numpy >= 1.20.0                 |
| opencv-python\~=4.8.0           |
| tflite-support>=0.4.2           |
| protobuf>=3.18.0,<4             |
| <p>NOTE::<br>- python 3.7.3</p> |
|                                 |
|                                 |
|                                 |
|                                 |

***

### Installation

| Original Tensorflow Video - 2 yrs ago                                                                                                                     | November 2023 - Recent Video                                                                                                                     |
| --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| [YouTube Link](https://youtube.com/watch?v=mNjXEybFn98\&list=PLQY2H8rRoyvz_anznBg6y3VhuSMcpN9oe)                                                          | [YouTube Link](https://www.youtube.com/watch?v=ldvlAMcGWIk)                                                                                      |
| <p>NOTE: (2:54)</p><p>-click top left</p><p>-preferences</p><p>-raspberry pi configuration</p><p>-tabs: interfaces</p><p>-enable camera</p><p>-reboot</p> | <p>NOTE: <br>-click top left</p><p>-preferences</p><p>-raspberry pi configuration</p><p>-tabs: interfaces</p><p>-enable camera</p><p>-reboot</p> |
| uaYq12tCsfz3-hWdK8ivKExIY                                                                                                                                 | uaYq12tCsfz3-5gal0Pt2blrl                                                                                                                        |

| sudo apt-get update                                                                                                                                                        |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| sudo apt-get install python3-tflite-runtime                                                                                                                                |
| git clone <https://github.com/tensorflow/examples.git> --depth 1                                                                                                           |
| cd example/object\_detection/raspberrypi/                                                                                                                                  |
| NOTE: change opencv version                                                                                                                                                |
| nano requirements.txt                                                                                                                                                      |
| opencv-python\~=4.8.0                                                                                                                                                      |
| <p>#from tflite official pg<br><br>bash <a href="https://github.com/tensorflow/examples/blob/master/lite/examples/object_detection/raspberry_pi/setup.sh">setup.sh</a></p> |
| pip install protobuf==3.20.\*                                                                                                                                              |
| python -m pip install --upgrade tflite-support==0.4.2           #didnt download                                                                                            |
| python -m pip install --upgrade tflite-runtime==2.11.0          #didnt download                                                                                            |
| pip install numpy==1.23                                                                                                                                                    |
|                                                                                                                                                                            |
| <p>(33:16)<br>opens detect.py-in thonny and then edits custom model and then runs it to detect<br>#Python 3.10.12</p>                                                      |

| <p>(3:26)<br>sudo apt-get update</p>                      |
| --------------------------------------------------------- |
| python3 --version                                         |
| NOTE:: python 3.7.3                                       |
| python3 -m pip install --user --upgrade pip               |
| python3 -m pip install --user virtualenv                  |
| python -m venv \~/tflite                                  |
| source \~/tflite/bin/activate                             |
| git clone <https://github.com/tensorflow/examples.git>    |
| cd examples/lite/examples/object\_detection/raspberry\_pi |
| sh setup.sh                                               |
| python detect.py                                          |
|                                                           |
|                                                           |
|                                                           |
|                                                           |
|                                                           |

### 5 - Run on Pi

| Original Tensorflow Video - 2 yrs ago                      |
| ---------------------------------------------------------- |
| source tflite/bin/activate                                 |
| cd examples/lite/examples/object\_detection/raspberry\_pi/ |
| cp \~/Downloads/android.tflite .                           |
| python detect.py                                           |
| python detect.py --model android.tflite                    |

### Sources

\[1] [Tensorflow Video](https://youtu.be/-ZyFYniGUsw)[b](https://colab.research.google.com/drive/1wRkzzI_MYYkbFGcRtP60NbSDyYgrJiyw?usp=sharing) (The Colab they have doesn't work with newer Python versions)