mediapipe: Seems that face detection python solution doesn't respect min_detection_confidence parameter

Please make sure that this is a solution issue.

System information (Please provide as much relevant information as possible)

  • Have I written custom code (as opposed to using a stock example script provided in Mediapipe): No
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04, Android 11, iOS 14.4): Windows 10 (OS Build 19045.2251)
  • MediaPipe version: 0.9.0
pip show mediapipe
Name: mediapipe
Version: 0.9.0
Summary: MediaPipe is the simplest way for researchers and developers to build world-class ML solutions and applications for mobile, edge, cloud and the web.
Home-page: https://github.com/google/mediapipe
Author: The MediaPipe Authors
Author-email: mediapipe@google.com
License: Apache 2.0
Location: c:\python310\lib\site-packages
Requires: absl-py, attrs, flatbuffers, matplotlib, numpy, opencv-contrib-python, protobuf
Required-by:
  • Bazel version: 5.2.0
  • Solution (e.g. FaceMesh, Pose, Holistic): FaceDetection
  • Programming Language and version ( e.g. C++, Python, Java): Python

Describe the expected behavior:

Here is the code I’m trying to execute:

Python 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import mediapipe as mp
>>> mp_face_detection = mp.solutions.face_detection
>>> face_detection = mp_face_detection.FaceDetection(model_selection=1, min_detection_confidence=0.3)

When I check graph config for this solution (face_detection._graph.text_config) I’m getting the following:

node {
  name: "facedetectionfullrange__facedetection__ToImageCalculator"
  calculator: "ToImageCalculator"
  input_stream: "IMAGE:image"
  output_stream: "IMAGE:facedetectionfullrange__facedetection__multi_backend_image"
}
node {
  name: "facedetectionfullrange__facedetection__ImageToTensorCalculator"
  calculator: "ImageToTensorCalculator"
  input_stream: "IMAGE:facedetectionfullrange__facedetection__multi_backend_image"
  output_stream: "TENSORS:facedetectionfullrange__facedetection__input_tensors"
  output_stream: "MATRIX:facedetectionfullrange__facedetection__transform_matrix"
  node_options {
    [type.googleapis.com/mediapipe.ImageToTensorCalculatorOptions] {
      output_tensor_width: 192
      output_tensor_height: 192
      keep_aspect_ratio: true
      output_tensor_float_range {
        min: -1
        max: 1
      }
      border_mode: BORDER_ZERO
    }
  }
}
node {
  name: "facedetectionfullrange__facedetection__SsdAnchorsCalculator"
  calculator: "SsdAnchorsCalculator"
  output_side_packet: "facedetectionfullrange__facedetection__anchors"
  node_options {
    [type.googleapis.com/mediapipe.SsdAnchorsCalculatorOptions] {
      input_size_width: 192
      input_size_height: 192
      min_scale: 0.1484375
      max_scale: 0.75
      anchor_offset_x: 0.5
      anchor_offset_y: 0.5
      num_layers: 1
      strides: 4
      aspect_ratios: 1
      interpolated_scale_aspect_ratio: 0
      fixed_anchor_size: true
    }
  }
}
node {
  name: "facedetectionfullrange__facedetection__inferencecalculator__facedetectionfullrange__facedetection__InferenceCalculator"
  calculator: "InferenceCalculatorCpu"
  input_stream: "TENSORS:facedetectionfullrange__facedetection__input_tensors"
  output_stream: "TENSORS:facedetectionfullrange__facedetection__detection_tensors"
  node_options {
    [type.googleapis.com/mediapipe.InferenceCalculatorOptions] {
      model_path: "mediapipe/modules/face_detection/face_detection_full_range_sparse.tflite"
      delegate {
        xnnpack {
        }
      }
    }
  }
}
node {
  name: "facedetectionfullrange__facedetection__TensorsToDetectionsCalculator"
  calculator: "TensorsToDetectionsCalculator"
  input_stream: "TENSORS:facedetectionfullrange__facedetection__detection_tensors"
  output_stream: "DETECTIONS:facedetectionfullrange__facedetection__unfiltered_detections"
  input_side_packet: "ANCHORS:facedetectionfullrange__facedetection__anchors"
  node_options {
    [type.googleapis.com/mediapipe.TensorsToDetectionsCalculatorOptions] {
      num_classes: 1
      num_boxes: 2304
      num_coords: 16
      x_scale: 192
      y_scale: 192
      w_scale: 192
      h_scale: 192
      keypoint_coord_offset: 4
      num_keypoints: 6
      num_values_per_keypoint: 2
      box_coord_offset: 0
      reverse_output_order: true
      sigmoid_score: true
      score_clipping_thresh: 100
      min_score_thresh: 0.6
    }
  }
}
node {
  name: "facedetectionfullrange__facedetection__NonMaxSuppressionCalculator"
  calculator: "NonMaxSuppressionCalculator"
  input_stream: "facedetectionfullrange__facedetection__unfiltered_detections"
  output_stream: "facedetectionfullrange__facedetection__filtered_detections"
  options {
    [mediapipe.NonMaxSuppressionCalculatorOptions.ext] {
      min_suppression_threshold: 0.3
      overlap_type: INTERSECTION_OVER_UNION
      algorithm: WEIGHTED
    }
  }
}
node {
  name: "facedetectionfullrange__facedetection__DetectionProjectionCalculator"
  calculator: "DetectionProjectionCalculator"
  input_stream: "DETECTIONS:facedetectionfullrange__facedetection__filtered_detections"
  input_stream: "PROJECTION_MATRIX:facedetectionfullrange__facedetection__transform_matrix"
  output_stream: "DETECTIONS:detections"
}
input_stream: "IMAGE:image"
executor {
}
output_stream: "DETECTIONS:detections"
type: "FaceDetectionFullRangeCpu"
graph_options {
  [type.googleapis.com/mediapipe.FaceDetectionOptions] {
    min_score_thresh: 0.3
  }
}

As I can see configured min_detection_confidence is added as graph_options field min_score_thresh in CalculatorGraphConfig and I suppose it should be mapped to any calculator options using option_value: syntax as described here, but I don’t see it (I expected that it should be mapped to min_score_thresh in TensorsToDetectionsCalculatorOptions). So when I try to change min_detection_confidence parameter for the solution I always get the same output and this parameter looks like doesn’t affect anything. But I suppose it should.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 16

Most upvoted comments

Hello @kuaashish,

Verified that the new solution doesn’t have this issue.

Thanks

Hello @smozhaykin This is to provide an update. The issue has been discussed internally and we shall be picking it up for a fix.