Object detection using YOLO on Edge TPU

[eunjins]

Hi,

I want to run this code.
detect_image.py

The original recommendation is the ssd-mobilenet model, but I put the yolov4-tiny model I compiled.

$ python3 examples/detect_image.py \
  --model test_data/yolov4-tiny-relu-new_coords-int8_edgetpu.tflite \
  --labels test_data/coco_labels.txt \
  --input test_data/grace_hopper.bmp

----INFERENCE TIME----
Note: The first inference is slow because it includes loading the model into Edge TPU memory.
Traceback (most recent call last):
  File "examples/detect_image.py", line 108, in <module>
    main()
  File "examples/detect_image.py", line 87, in main
    objs = detect.get_objects(interpreter, args.threshold, scale)
  File "/usr/lib/python3/dist-packages/pycoral/adapters/detect.py", line 191, in get_objects
    scores = common.output_tensor(interpreter, 2)[0]
  File "/usr/lib/python3/dist-packages/pycoral/adapters/common.py", line 29, in output_tensor
    return interpreter.tensor(interpreter.get_output_details()[i]['index'])()
IndexError: list index out of range

I compiled tflite with Edge TPU compiler like this. (Classification works fine for this model, only object detection doesn't work.)
My colab

I can't find what the problem is. Is the model compilation wrong? Or do I need another code?
Thanks

[Fetulhak]

@eunjins how can i get prediction results of my image. I mean the [x,y,w,h,cof,cls]

[JayantGoel001]

@eunjins how can i get prediction results of my image. I mean the [x,y,w,h,cof,cls]

Hello @Fetulhak
You can use this to get the result of the image.

bboxes = yolo.predict(frame,prob_thresh=prob_thresh)
here, frame is image array,prob_thresh is desired threshold

bboxes will be an array where each element having 6 parameters
1,2,3,4 will Coordinates
5th will be class ID
6th will be a probability.

[Fetulhak]

@eunjins how can i get prediction results of my image. I mean the [x,y,w,h,cof,cls]

Hello @Fetulhak
You can use this to get the result of the image.

bboxes = yolo.predict(frame,prob_thresh=prob_thresh)
here, frame is image array,prob_thresh is desired threshold

bboxes will be an array where each element having 6 parameters
1,2,3,4 will Coordinates
5th will be class ID
6th will be a probability.

Thanks.

[Fetulhak]

@eunjins at the baseclass here https://github.com/hhk7734/tensorflow-yolov4/blob/master/py_src/yolov4/common/base_class.py

the predict class is defined like this

def predict(self, frame: np.ndarray, prob_thresh: float):
    # pylint: disable=unused-argument, no-self-use
    return [[0.0, 0.0, 0.0, 0.0, -1]]

isn't is 6 things should it return like you explain it above but from the return value it is only five

[JayantGoel001]

@eunjins at the baseclass here https://github.com/hhk7734/tensorflow-yolov4/blob/master/py_src/yolov4/common/base_class.py

the predict class is defined like this

def predict(self, frame: np.ndarray, prob_thresh: float):
    # pylint: disable=unused-argument, no-self-use
    return [[0.0, 0.0, 0.0, 0.0, -1]]

isn't is 6 things should it return like you explain it above but from the return value it is only five

Check this file
https://github.com/hhk7734/tensorflow-yolov4/blob/master/py_src/yolov4/tf/__init__.py

[hhk7734]

@Fetulhak
BaseClass.predict is just abstract method. I will modify it later.

tensorflow-yolov4/py_src/yolov4/tflite/__init__.py

Lines 131 to 140 in 3d31292

	def predict(self, frame: np.ndarray, prob_thresh: float) -> np.ndarray:
	"""
	Predict one frame
	@param `frame`: Dim(height, width, channels)
	@param `prob_thresh`
	@return pred_bboxes
	Dim(-1, (x, y, w, h, cls_id, prob))
	"""

[hhk7734]

@eunjins

Number of Edge TPU subgraphs: 1
Total number of operations: 49

This is realy interesting. Would you share your tflite file?

[eunjins]

@hhk7734 Here is the tflite file.
yolov4-tiny-relu-new_coords-int8_edgetpu.zip

[eunjins]

@Fetulhak @JayantGoel001
Should I change the return value? I'm confused..

[hhk7734]

@eunjins
why do you want run detect_image.py?
it looks like the below script.

https://wiki.loliot.net/docs/lang/python/libraries/yolov4/python-yolov4-edge-tpu#run-example-script

[eunjins]

@hhk7734
In fact, the purpose is to detect objects.
I wanted to apply the code I wrote to evaluate the model to yolo, so I tried running detect_image.py.
So is my tflite file ok?

[Fetulhak]

@hhk7734 when i use "pip install -U yolov4" in google colab it only mount the "py_src/yolov4" part of the github. how to pip install the whole github directory so that we can access the "coco.names" under test folder and the whole project as it is in the github

[Fetulhak]

@hhk7734 @eunjins @JayantGoel001 when I run the demo on the documentation after pip install yolov4

import cv2

from yolov4.tf import YOLOv4

yolo = YOLOv4()

yolo.config.parse_names("coco.names")
yolo.config.parse_cfg("yolov4-tiny.cfg")

yolo.make_model()
yolo.load_weights("yolov4-tiny.weights", weights_type="yolo")
yolo.summary(summary_type="yolo")
yolo.summary()

yolo.inference(media_path="kite.jpg")

yolo.inference(media_path="road.mp4", is_image=False)

yolo.inference(
"/dev/video0",
is_image=False,
cv_apiPreference=cv2.CAP_V4L2,
cv_frame_size=(640, 480),
cv_fourcc="YUYV",
)

I have the following error:

/usr/local/lib/python3.7/dist-packages/yolov4/common/parser.py in parse_names(names_path)
130 """
131 names: Dict[int, str] = {}
--> 132 with open(names_path, "r") as fd:
133 index = 0
134 for class_name in fd:

FileNotFoundError: [Errno 2] No such file or directory: 'coco.names'

[Fetulhak]

@hhk7734 @eunjins @JayantGoel001 when I run the demo on the documentation after pip install yolov4

import cv2

from yolov4.tf import YOLOv4

yolo = YOLOv4()

yolo.config.parse_names("coco.names")
yolo.config.parse_cfg("yolov4-tiny.cfg")

yolo.make_model()
yolo.load_weights("yolov4-tiny.weights", weights_type="yolo")
yolo.summary(summary_type="yolo")
yolo.summary()

yolo.inference(media_path="kite.jpg")

yolo.inference(media_path="road.mp4", is_image=False)

yolo.inference(
"/dev/video0",
is_image=False,
cv_apiPreference=cv2.CAP_V4L2,
cv_frame_size=(640, 480),
cv_fourcc="YUYV",
)

I have the following error:

/usr/local/lib/python3.7/dist-packages/yolov4/common/parser.py in parse_names(names_path)
130 """
131 names: Dict[int, str] = {}
--> 132 with open(names_path, "r") as fd:
133 index = 0
134 for class_name in fd:

FileNotFoundError: [Errno 2] No such file or directory: 'coco.names'

Now I solve the errno 2 problem by mounting my drive to google colab and putting those three files "cco.names", "yolov4-tiny.cfg" , "yolov4-tiny.weights" and "kite.gpg" in my drive and giving that directory for the inference() function

here is the sample code any one who gets into the same trouble

import cv2

from yolov4.tf import YOLOv4

yolo = YOLOv4()

yolo.config.parse_names("/content/drive/MyDrive/coco.names")
yolo.config.parse_cfg("/content/drive/MyDrive/yolov4-tiny.cfg")

yolo.make_model()
yolo.load_weights("/content/drive/MyDrive/yolov4-tiny.weights", weights_type="yolo")
yolo.summary(summary_type="yolo")
yolo.summary()
#cv2.imread("/content/drive/MyDrive/kite.jpg")
yolo.inference(media_path="/content/drive/MyDrive/kite.jpg")

Okay there is another error this time. the problem now is the inference function definition>>>which brings me back to my previous question #86 (comment) which is the predict function inside base class

here is how inference function is defined

def inference(
    self,
    media_path,
    is_image: bool = True,
    cv_apiPreference=None,
    cv_frame_size: tuple = None,
    cv_fourcc: str = None,
    cv_waitKey_delay: int = 1,
    prob_thresh: float = 0.25,
):
    if isinstance(media_path, str) and not path.exists(media_path):
        raise FileNotFoundError("{} does not exist".format(media_path))

    cv2.namedWindow("result", cv2.WINDOW_AUTOSIZE)

    if is_image:
        frame = cv2.imread(media_path)
        frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

        start_time = time.time()
        bboxes = self.predict(frame_rgb, prob_thresh=prob_thresh)

my question now is isn't it self.predict going to call the predict function in the base class which is wrongly defined?????

[JayantGoel001]

@hhk7734 @eunjins @JayantGoel001 when I run the demo on the documentation after pip install yolov4
import cv2
from yolov4.tf import YOLOv4
yolo = YOLOv4()
yolo.config.parse_names("coco.names")
yolo.config.parse_cfg("yolov4-tiny.cfg")
yolo.make_model()
yolo.load_weights("yolov4-tiny.weights", weights_type="yolo")
yolo.summary(summary_type="yolo")
yolo.summary()
yolo.inference(media_path="kite.jpg")
yolo.inference(media_path="road.mp4", is_image=False)
yolo.inference(
"/dev/video0",
is_image=False,
cv_apiPreference=cv2.CAP_V4L2,
cv_frame_size=(640, 480),
cv_fourcc="YUYV",
)
I have the following error:
/usr/local/lib/python3.7/dist-packages/yolov4/common/parser.py in parse_names(names_path)
130 """
131 names: Dict[int, str] = {}
--> 132 with open(names_path, "r") as fd:
133 index = 0
134 for class_name in fd:
FileNotFoundError: [Errno 2] No such file or directory: 'coco.names'

Now I solve the errno 2 problem by mounting my drive to google colab and putting those three files "cco.names", "yolov4-tiny.cfg" , "yolov4-tiny.weights" and "kite.gpg" in my drive and giving that directory for the inference() function

here is the sample code any one who gets into the same trouble

import cv2

from yolov4.tf import YOLOv4

yolo = YOLOv4()

yolo.config.parse_names("/content/drive/MyDrive/coco.names")
yolo.config.parse_cfg("/content/drive/MyDrive/yolov4-tiny.cfg")

yolo.make_model()
yolo.load_weights("/content/drive/MyDrive/yolov4-tiny.weights", weights_type="yolo")
yolo.summary(summary_type="yolo")
yolo.summary()
#cv2.imread("/content/drive/MyDrive/kite.jpg")
yolo.inference(media_path="/content/drive/MyDrive/kite.jpg")

Okay there is another error this time. the problem now is the inference function definition>>>which brings me back to my previous question #86 (comment) which is the predict function inside base class

here is how inference function is defined

def inference(
    self,
    media_path,
    is_image: bool = True,
    cv_apiPreference=None,
    cv_frame_size: tuple = None,
    cv_fourcc: str = None,
    cv_waitKey_delay: int = 1,
    prob_thresh: float = 0.25,
):
    if isinstance(media_path, str) and not path.exists(media_path):
        raise FileNotFoundError("{} does not exist".format(media_path))

    cv2.namedWindow("result", cv2.WINDOW_AUTOSIZE)

    if is_image:
        frame = cv2.imread(media_path)
        frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

        start_time = time.time()
        bboxes = self.predict(frame_rgb, prob_thresh=prob_thresh)

my question now is isn't it self.predict going to call the predict function in the base class which is wrongly defined?????

Hi @Fetulhak
The answer to your question is NO.
Actually, You are looking at the wrong file
Look into this file https://github.com/hhk7734/tensorflow-yolov4/blob/master/py_src/yolov4/tf/__init__.py
we have imported like this yolov4.tf which will point to this file.⬆

Or the best thing you can do is try, debug and run till you fully understand the working of yolov4-tf.🙂

[JayantGoel001]

@hhk7734 when i use "pip install -U yolov4" in google colab it only mount the "py_src/yolov4" part of the github. how to pip install the whole github directory so that we can access the "coco.names" under test folder and the whole project as it is in the github

For this query instead of installing yolov4, you can clone this repo in your colab.
You can use the following command
!git clone https://github.com/hhk7734/tensorflow-yolov4.git
Then run the python file accordingly.