Thursday, December 7, 2017

[Tensorflow] Object Detection API test in Python and TensorflowSharp

py_tensorflow_2
Object detection in machine learning is a popular topic, this time I will test Tensorflow Object Detection API in Python(official) and TensorflowSharp(wrapper) and check the result.

Let's start with Python, my code was modify from Object Detection Demo with some simplify, avoid path/protobuf issue, hope this simple example can help more people successful test it.


Here is some code explain, I will only list the part that I have modify.

Import

import numpy as np
import os
import tensorflow as tf
import cv2
import re
import time

if tf.__version__ != '1.4.0':
  raise ImportError('Please upgrade your tensorflow installation to v1.4.0!')
Use opencv replace matlib, add re for read labels, remove the download related package.


Model

# What model to download.
MODEL_NAME = 'ssd_mobilenet_v1_coco_2017_11_17'
# Path to frozen detection graph. This is the actual model that is used for the object detection.
PATH_TO_CKPT = MODEL_NAME + '/frozen_inference_graph.pb'
# List of the strings that is used to add correct label for each box.
PATH_TO_LABELS = 'mscoco_label_map.pbtxt'
I removed auto model download, so you have to download it from Tensorflow detection model zoo.


Loading label map

def load_label(file):
 with open(file,'r') as fi:
  match = re.finditer('item {\n  name: \"(?P.*)\"\n  id: (?P\d+)\n  display_name: \"(?P
using regular expression to load label map as a list.


Visualization result

      # Visualization of the results of a detection.
      for i in range(0,int(num[0])):
        confidence = scores[0,i]
        if confidence>0.5:
            idx = int(classes[0,i])
            box = boxes[0, i, 0:4] * np.array([h, w, h, w])
            (y, x, y2, x2) = box.astype("int")
            # draw rect
            cv2.rectangle(image, (x, y), (x2, y2), colors[idx], 2)
            # draw label
            label = "{}: {:.2f}%".format([t['label'] for t in labels if t['id'] == str(idx)][0], confidence * 100)
            print("{}".format(label))
            (fontX, fontY) = cv2.getTextSize(label, cv2.FONT_HERSHEY_TRIPLEX, 0.5, 1)[0]
            y = y + fontY if y-fontY<0 else y
            cv2.rectangle(image,(x, y-fontY),(x+fontX, y),colors[idx],cv2.FILLED)
            cv2.putText(image, label, (x, y), cv2.FONT_HERSHEY_TRIPLEX, 0.5, (0,0,0), 1)
      cv2.imshow(image_path,image)
Draw result on test image like my previous post doing.


By default, this example using "ssd_mobilenet_v1_coco_2017_11_17" model, the result should be like that.
cs_tensorflow_4

py_tensorflow_5


TensorflowSharp

Same as python, in C# I modify code from ExampleObjectDetection, dunno why in tensorflowSharp load image input to model are way complex than load in tensorflow-python, even I try to simplify it, the code still long and unfriendly, so I will not post code on here, but you still can check it on my github.

Anyway, one thing to be noticed, the detect result in tensorflow-python and TensorflowSharp are different.
snip_20171206174559
Using same model: "ssd_mobilenet_v1_coco_2017_11_17", in TensorflowSharp you can't get same result as official tensorflow, so if you want to develop in TensorflowSharp, be careful with it!!


Better result

You can try change model from Tensorflow detection model zoo to check the result, if using faster_rcnn_inception_resnet_v2_atrous_coco you can get better detection result like Tensorflow Object Detection API showed.

py_tensorflow_2

With the better detection result, the run time will increase a huge amount, on my pc the runtime will increase about 100 times(using CPU calculated).


Source Code

Here is my python's full code.
import numpy as np
import os
import tensorflow as tf
import cv2
import re
import time

if tf.__version__ != '1.4.0':
  raise ImportError('Please upgrade your tensorflow installation to v1.4.0!')

# What model to download.
#MODEL_NAME = 'faster_rcnn_inception_resnet_v2_atrous_coco_11_06_2017'
MODEL_NAME = 'ssd_mobilenet_v1_coco_2017_11_17'
# Path to frozen detection graph. This is the actual model that is used for the object detection.
PATH_TO_CKPT = MODEL_NAME + '/frozen_inference_graph.pb'
# List of the strings that is used to add correct label for each box.
PATH_TO_LABELS = 'mscoco_label_map.pbtxt'

detection_graph = tf.Graph()
with detection_graph.as_default():
  od_graph_def = tf.GraphDef()
  with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid:
    serialized_graph = fid.read()
    od_graph_def.ParseFromString(serialized_graph)
    tf.import_graph_def(od_graph_def, name='')

def load_image_into_numpy_array(image):
  (im_width, im_height) = image.size
  return np.array(image.getdata()).reshape(
      (im_height, im_width, 3)).astype(np.uint8)

def load_label(file):
 with open(file,'r') as fi:
  match = re.finditer('item {\n  name: \"(?P.*)\"\n  id: (?P\d+)\n  display_name: \"(?P

All code will put on github.



BTW, here has another Tensorflow detection model zoo with more models, you can try it if need.

4 comments:

  1. There's a bug with the tensorflowsharp code that gives you incorrect results. The fix is:
    const float scale = 1;
    to
    const float scale = 128;

    ReplyDelete
    Replies
    1. What is now the best value for scale ?
      With 128 the result is getting worse (no kite recognized at all) than with 1 (3 kites detected with low percentage, as above).

      Delete
  2. Could you get the .NET sample working with TensorFlowSharp version 1.5.0 (works only with version 1.4.0) ?
    The following runtime exception is thrown in this case:

    Error: untreated exception: System.DllNotFoundException: could not load DLL 'libtensorflow': the specified module could not be found.

    ReplyDelete