pylibdmtx: BUG: Rect object returned by `decode` can be trivially wrong

The rectangle object returned by the decode function is (completely) wrong whenever the found DataMatrix code is rotated. The problem is that the returned rectangle is always supposed to be upright when, in fact, libdtmx supports also rotated codes.

Examples with one upright and one rotated DataMatrix code:

  • direct call of dmtxread on the command line
(base) ➜  ~ dmtxread -n -v -N 1 12345_garbage.png        
--------------------------------------------------
       Matrix Size: 10 x 10
    Data Codewords: 3 (capacity 3)
   Error Codewords: 5
      Data Regions: 1 x 1
Interleaved Blocks: 1
    Rotation Angle: 0
          Corner 0: (1152.0, 1287.0)
          Corner 1: (1201.0, 1287.0)
          Corner 2: (1200.1, 1238.0)
          Corner 3: (1152.0, 1238.0)
--------------------------------------------------
12345

(base) ➜  ~ dmtxread -n -v -N 1 12345_garbage_rotated.png
--------------------------------------------------
       Matrix Size: 10 x 10
    Data Codewords: 3 (capacity 3)
   Error Codewords: 5
      Data Regions: 1 x 1
Interleaved Blocks: 1
    Rotation Angle: 319
          Corner 0: (1191.0, 1292.9)
          Corner 1: (1228.1, 1325.2)
          Corner 2: (1260.5, 1287.9)
          Corner 3: (1223.4, 1255.6)
--------------------------------------------------
12345
  • using pylibdtmx
(libdmtx) ➜  ~ ipython
Python 3.7.5 (default, Oct 25 2019, 15:51:11) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.9.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from pylibdmtx.pylibdmtx import decode                                                                           

In [2]: from PIL import Image                                                                                            

In [3]: decode(Image.open("/home/gri/12345_garbage.png"), max_count=1)                                                   
Out[3]: [Decoded(data=b'12345', rect=Rect(left=1152, top=2220, width=48, height=49))]

In [4]: decode(Image.open("/home/gri/12345_garbage_rotated.png"), max_count=1)                                           
Out[4]: [Decoded(data=b'12345', rect=Rect(left=1191, top=2214, width=70, height=5))]

(Note how also width and height changed to totally incorrect values.)

The obvious solution would be to not return a tuple of left, top, width, and height coordinates, since they apply only to upright rectangles, but to return the coordinates of the four corners—as is done by the command line program.

So in these lines of the code all four points should be captured instead of only two ones.

Of course, such a change would break the interface as it is now. If you insist on returning an upright rectangle, it should at least be the upright bounding box of the DataMatrix code crop. But I would much prefer if the python version would just return the four corner points as the command line version does.

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Reactions: 3
  • Comments: 15

Most upvoted comments

yes you can do by easily adding two more vectors e.g p00 = DmtxVector2() p11 = DmtxVector2(1.0, 1.0) p10 = DmtxVector2(1.0, 0.0) p01 = DmtxVector2(0.0, 1.0) dmtxMatrix3VMultiplyBy(p00, region.contents.fit2raw) dmtxMatrix3VMultiplyBy(p11, region.contents.fit2raw) dmtxMatrix3VMultiplyBy(p01, region.contents.fit2raw) dmtxMatrix3VMultiplyBy(p10, region.contents.fit2raw) x00 = int((shrink * p00.X) + 0.5) y00 = int((shrink * p00.Y) + 0.5) x11 = int((shrink * p11.X) + 0.5) y11 = int((shrink * p11.Y) + 0.5) x10 = int((shrink * p10.X) + 0.5) y10 = int((shrink * p10.Y) + 0.5) x01 = int((shrink * p01.X) + 0.5) y01 = int((shrink * p01.Y) + 0.5) return Decoded( string_at(msg.contents.output), #Rect2 = namedtuple(‘Rect’, ‘x00 y00 x01 y01 x10 y10 x11 y11’) Rect2(x00,y00,x01,y01,x10,y10,x11,y11)

I did its in my lib and it works well

Few months late to this but I recently needed this functionality, strange how they haven’t merged it in yet. I didn’t want to modify the actual library so here is my implementation, thanks to @jepperaskdk and @amackp:

from pylibdmtx.pylibdmtx import _region, _decoder, _image, _pixel_data, _decoded_matrix_region
from pylibdmtx.wrapper import c_ubyte_p, DmtxPackOrder, DmtxVector2, dmtxMatrix3VMultiplyBy, DmtxUndefined
from ctypes import cast, string_at
from collections import namedtuple
import numpy

_pack_order = {
    8: DmtxPackOrder.DmtxPack8bppK,
    16: DmtxPackOrder.DmtxPack16bppRGB,
    24: DmtxPackOrder.DmtxPack24bppRGB,
    32: DmtxPackOrder.DmtxPack32bppRGBX,
}
Decoded = namedtuple('Decoded', 'data rect')


def decode_with_region(image):
    results = []
    pixels, width, height, bpp = _pixel_data(image)
    with _image(cast(pixels, c_ubyte_p), width, height, _pack_order[bpp]) as img:
        with _decoder(img, 1) as decoder:
            while True:
                with _region(decoder, None) as region:
                    if not region:
                        break
                    else:
                        res = _decode_region(decoder, region)
                        if res:
                            open_cv_image = numpy.array(image)
                            # Convert RGB to BGR
                            open_cv_image = open_cv_image[:, :, ::-1].copy()
                            height, width, _ = open_cv_image.shape

                            topLeft = (res.rect['01']['x'], height - res.rect['01']['y'])
                            topRight = (res.rect['11']['x'], height - res.rect['11']['y'])
                            bottomRight = (res.rect['10']['x'], height - res.rect['10']['y'])
                            bottomLeft = (res.rect['00']['x'], height - res.rect['00']['y'])
                            results.append(Decoded(res.data, (topLeft, topRight, bottomRight, bottomLeft)))
    return results


def _decode_region(decoder, region):
    with _decoded_matrix_region(decoder, region, DmtxUndefined) as msg:
        if msg:
            vector00 = DmtxVector2()
            vector11 = DmtxVector2(1.0, 1.0)
            vector10 = DmtxVector2(1.0, 0.0)
            vector01 = DmtxVector2(0.0, 1.0)
            dmtxMatrix3VMultiplyBy(vector00, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(vector11, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(vector01, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(vector10, region.contents.fit2raw)

            return Decoded(
                string_at(msg.contents.output),
                {
                    '00': {
                        'x': int((vector00.X) + 0.5),
                        'y': int((vector00.Y) + 0.5)
                    },
                    '01': {
                        'x': int((vector01.X) + 0.5),
                        'y': int((vector01.Y) + 0.5)
                    },
                    '10': {
                        'x': int((vector10.X) + 0.5),
                        'y': int((vector10.Y) + 0.5)
                    },
                    '11': {
                        'x': int((vector11.X) + 0.5),
                        'y': int((vector11.Y) + 0.5)
                    }
                }
            )
        else:
            return None

You are right - I flipped the image. That’s the only solution I’ve found

`while camera.IsGrabbing(): grabResult = camera.RetrieveResult(5000, pylon.TimeoutHandling_ThrowException) t0 = time.time() if grabResult.GrabSucceeded(): image = converter.Convert(grabResult) img = image.GetArray() img_flip = cv2.flip(img, 0) codes = decode(img, timeout=8000, max_count=99, corrections=3)

do something

cv2.imshow(‘dm’, img_flip)`

Solved!

How did you solve it?

@GenericArt Not sure what @Gennadynemchin 's actual issue was, but I found that the when using @gauti1311 's solution to return 4 corners from the _decode_region, the y coordinate frame is flipped relative to the coordinate frame used by openCV images. You can simply subtract the input image’s height from the rectangle’s Y coordinates to get the proper pixel location.

cv_image = cv2.imread('/path/to/image.png')

results = pylibdmtx.decode(cv_image)

height, width, channels = cv_image.shape

for code in results:
    tl = (code.rect2.x01, height - code.rect2.y01)
    tr = (code.rect2.x11, height - code.rect2.y11)
    br = (code.rect2.x10, height - code.rect2.y10)
    bl = (code.rect2.x00, height - code.rect2.y00)
   
    # do something with rectangle... 

Is there a reason why this bug is still present, 2-3 years after this issue was created?

@gauti1311’s solution works for me. @rehman922 you have to alter the pylibdmtx/pylibdmtx.py _decode_region function. Either return the 4 corners as @gauti1311 or get the boundingbox (preserving the use of Rect) like so:

def _decode_region(decoder, region, corrections, shrink):
    """Decodes and returns the value in a region.

    Args:
        region (DmtxRegion):

    Yields:
        Decoded or None: The decoded value.
    """
    with _decoded_matrix_region(decoder, region, corrections) as msg:
        if msg:
            # Coordinates
            p00 = DmtxVector2()
            p11 = DmtxVector2(1.0, 1.0)
            p10 = DmtxVector2(1.0, 0.0)
            p01 = DmtxVector2(0.0, 1.0)
            dmtxMatrix3VMultiplyBy(p00, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(p11, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(p01, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(p10, region.contents.fit2raw)
            x00 = int((shrink * p00.X) + 0.5)
            y00 = int((shrink * p00.Y) + 0.5)
            x11 = int((shrink * p11.X) + 0.5)
            y11 = int((shrink * p11.Y) + 0.5)
            x10 = int((shrink * p10.X) + 0.5)
            y10 = int((shrink * p10.Y) + 0.5)
            x01 = int((shrink * p01.X) + 0.5)
            y01 = int((shrink * p01.Y) + 0.5)

            min_x = min(x00, x11, x10, x01)
            max_x = max(x00, x11, x10, x01)
            min_y = min(y00, y11, y10, y01)
            max_y = max(y00, y11, y10, y01)

            return Decoded(
                string_at(msg.contents.output),
                Rect(min_x, min_y, max_x - min_x, min_y - max_y)
            )
        else:
            return None