opencv: cvtColorTwoPlane throwing Fatal signal 11 (SIGSEGV), code 2 (SEGV_ACCERR) error

  • OpenCV => 4.1.0
  • Operating System / Platform => Android arm
  • Compiler => .so library
Detailed description

Similar to JavaCamera2View, I am trying to convert the YUV_420_888 image to BGR Mat. I have modified the JavaCamera2View method (bgra) to following below. Imgproc.cvtColorTwoPlane(y_mat, uv_mat, mBgr, Imgproc.COLOR_YUV2BGR_NV21) is throwing error on my uvc camera based single board computer running android. It works fine on my android based mobile phone. Error logs are provided below.

Steps to reproduce
public Mat bgr() {
        Image.Plane[] planes = mImage.getPlanes();
        int w = mImage.getWidth();
        int h = mImage.getHeight();
        int chromaPixelStride = planes[1].getPixelStride();

        if (chromaPixelStride == 2) { // Chroma channels are interleaved
            ByteBuffer y_plane = planes[0].getBuffer();
            ByteBuffer uv_plane = planes[1].getBuffer();
            Mat y_mat = new Mat(h, w, CvType.CV_8UC1, y_plane);
            Mat uv_mat = new Mat(h / 2, w / 2, CvType.CV_8UC2, uv_plane);
            
Imgproc.cvtColorTwoPlane(y_mat, uv_mat, mBgr, Imgproc.COLOR_YUV2BGR_NV21); // ********* Crashes here ******

            return mBgr;
        } else { // Chroma channels are not interleaved
            byte[] yuv_bytes = new byte[w * (h + h / 2)];
            ByteBuffer y_plane = planes[0].getBuffer();
            ByteBuffer u_plane = planes[1].getBuffer();
            ByteBuffer v_plane = planes[2].getBuffer();

            y_plane.get(yuv_bytes, 0, w * h);

            int chromaRowStride = planes[1].getRowStride();
            int chromaRowPadding = chromaRowStride - w / 2;

            int offset = w * h;
            if (chromaRowPadding == 0) {
                // When the row stride of the chroma channels equals their width, we can copy
                // the entire channels in one go
                u_plane.get(yuv_bytes, offset, w * h / 4);
                offset += w * h / 4;
                v_plane.get(yuv_bytes, offset, w * h / 4);
            } else {
                // When not equal, we need to copy the channels row by row
                for (int i = 0; i < h / 2; i++) {
                    u_plane.get(yuv_bytes, offset, w / 2);
                    offset += w / 2;
                    if (i < h / 2 - 1) {
                        u_plane.position(u_plane.position() + chromaRowPadding);
                    }
                }
                for (int i = 0; i < h / 2; i++) {
                    v_plane.get(yuv_bytes, offset, w / 2);
                    offset += w / 2;
                    if (i < h / 2 - 1) {
                        v_plane.position(v_plane.position() + chromaRowPadding);
                    }
                }
            }

            Mat yuv_mat = new Mat(h + h / 2, w, CvType.CV_8UC1);
            yuv_mat.put(0, 0, yuv_bytes);
            Imgproc.cvtColor(yuv_mat, mBgr, Imgproc.COLOR_YUV2BGR_I420, 3);

            return mBgr;
            
        }
    }

Error Message:

Fatal signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0xb3678000 in tid 13437

2019-04-22 02:31:20.866 13449-13449/? A/DEBUG: ABI: 'arm'
2019-04-22 02:31:20.866 13449-13449/? A/DEBUG: pid: 12779, tid: 13437, name: Camera2HandlerT  >>> com.ss.retailapp <<<
2019-04-22 02:31:20.866 13449-13449/? A/DEBUG: signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0xb3678000
2019-04-22 02:31:20.866 13449-13449/? A/DEBUG:     r0  b3677fe1  r1  b51fc540  r2  b51fc550  r3  b51fc660
2019-04-22 02:31:20.866 13449-13449/? A/DEBUG:     r4  b51fc660  r5  b83fdb30  r6  b51fc6a0  r7  b51fc788
2019-04-22 02:31:20.866 13449-13449/? A/DEBUG:     r8  b51fc620  r9  000009e0  r10 00000040  r11 b51fc6e0
2019-04-22 02:31:20.866 13449-13449/? A/DEBUG:     ip  f45add8c  sp  b51fc4d0  lr  b51fc590  pc  d30021ba
2019-04-22 02:31:20.869 13449-13449/? A/DEBUG: backtrace:
2019-04-22 02:31:20.869 13449-13449/? A/DEBUG:     #00 pc 002e61ba  /data/app/com.ss.retailapp-3L43PIzkF5FQYmOhkCNBJg==/lib/arm/libopencv_java4.so
2019-04-22 02:31:20.869 13449-13449/? A/DEBUG:     #01 pc 0026d1f5  /data/app/com.ss.retailapp-3L43PIzkF5FQYmOhkCNBJg==/lib/arm/libopencv_java4.so
2019-04-22 02:31:20.869 13449-13449/? A/DEBUG:     #02 pc 0026d6fb  /data/app/com.ss.retailapp-3L43PIzkF5FQYmOhkCNBJg==/lib/arm/libopencv_java4.so
2019-04-22 02:31:20.869 13449-13449/? A/DEBUG:     #03 pc 0026d58b  /data/app/com.ss.retailapp-3L43PIzkF5FQYmOhkCNBJg==/lib/arm/libopencv_java4.so
2019-04-22 02:31:20.869 13449-13449/? A/DEBUG:     #04 pc 00787263  /data/app/com.ss.retailapp-3L43PIzkF5FQYmOhkCNBJg==/lib/arm/libopencv_java4.so (tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::local_wait_for_all(tbb::task&, tbb::task*)+266)
2019-04-22 02:31:20.869 13449-13449/? A/DEBUG:     #05 pc 00780695  /data/app/com.ss.retailapp-3L43PIzkF5FQYmOhkCNBJg==/lib/arm/libopencv_java4.so (tbb::internal::arena::process(tbb::internal::generic_scheduler&)+68)
2019-04-22 02:31:20.869 13449-13449/? A/DEBUG:     #06 pc 007840f1  /data/app/com.ss.retailapp-3L43PIzkF5FQYmOhkCNBJg==/lib/arm/libopencv_java4.so (tbb::internal::market::process(rml::job&)+32)
2019-04-22 02:31:20.869 13449-13449/? A/DEBUG:     #07 pc 00784d31  /data/app/com.ss.retailapp-3L43PIzkF5FQYmOhkCNBJg==/lib/arm/libopencv_java4.so (tbb::internal::rml::private_worker::run()+68)
2019-04-22 02:31:20.869 13449-13449/? A/DEBUG:     #08 pc 00784ce5  /data/app/com.ss.retailapp-3L43PIzkF5FQYmOhkCNBJg==/lib/arm/libopencv_java4.so (tbb::internal::rml::private_worker::thread_routine(void*)+4)
2019-04-22 02:31:20.869 13449-13449/? A/DEBUG:     #09 pc 00063c35  /system/lib/libc.so (__pthread_start(void*)+22)
2019-04-22 02:31:20.869 13449-13449/? A/DEBUG:     #10 pc 0001e085  /system/lib/libc.so (__start_thread+22)

Edit: Image plane information updated below.

ImageImage object values: getFormat(): 35 , getWidth(): 2560 , getHeight(): 960
planes[0] : getPixelStride(): 1 , getRowStride(): 2560 , getBuffer().isDirect(): true
planes[1] : getPixelStride(): 2 , getRowStride(): 2560 , getBuffer().isDirect(): true
planes[2] : getPixelStride(): 2 , getRowStride(): 2560 , getBuffer().isDirect(): true

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 20 (7 by maintainers)

Most upvoted comments

Please try to “capture” well-known image, like this (in case of issues with color) and this (in case of issues with geometry).

Android canvas expects RGBA (not RGB) - problem may be there too. It is better to save image into PNG file via imwrite() (but it expects BGR on input, so R/G color will be swapped, but stride issue should be reproduced properly).

Also it would be nice to dump these buffers into RAW files from well known image:

  • planes[i].getBuffer() (i=0…2)

I have changed the Mat to RGBA before the Bitmap.createBitmap step.

I have attached the screenshot below as well as the buffer data for all 3 planes. From the image it looks like each of the 2 channels (U and V), gets compressed into one half of the image, one covers the top half and other the bottom half, giving impression of 3x image.

ImageTest.zip

image

@txdat You have to look at the frame data. I believe in my case, culprit was the camera hardware, as at certain resolution, it was giving incomplete data, so I had to ignore the last two pixel rows. The below code worked fine for my case.

int rowCut = 2; // Define rowCut size according to your actual image data
boolean flipImage = true;

private void createBgr(){
        if (mImage != null) {
            Image.Plane[] planes = mImage.getPlanes();

            int w = mImage.getWidth();
            int h = mImage.getHeight();
            int chromaPixelStride = planes[1].getPixelStride();

            if (chromaPixelStride == 2) { // Chroma channels are interleaved
                ByteBuffer y_plane = planes[0].getBuffer();
                ByteBuffer uv_plane = planes[1].getBuffer();
                Mat y_mat = new Mat(h, w, CvType.CV_8UC1, y_plane);
                Mat uv_mat = new Mat(h / 2, w / 2, CvType.CV_8UC2, uv_plane);
                // Changes for stereo camera and N2 //https://github.com/opencv/opencv/issues/14477
                Mat y_mat_cut = new Mat(y_mat, new Rect(0, 0, y_mat.cols(), y_mat.rows() - rowCut)); // cut 2 last rows from Y plane
                Mat uv_mat_cut = new Mat(uv_mat, new Rect(0, 0, uv_mat.cols(), uv_mat.rows() - rowCut / 2));
                Imgproc.cvtColorTwoPlane(y_mat_cut, uv_mat_cut, mBgr, Imgproc.COLOR_YUV2BGR_NV12); // With NV21 colours are inverted
                if (flipImage) {
                    Core.flip(mBgr, flipmBgrMat, 1);
                }
            } else { // Chroma channels are not interleaved
                byte[] yuv_bytes = new byte[w * (h + h / 2)];
                ByteBuffer y_plane = planes[0].getBuffer();
                ByteBuffer u_plane = planes[1].getBuffer();
                ByteBuffer v_plane = planes[2].getBuffer();

                y_plane.get(yuv_bytes, 0, w * h);

                int chromaRowStride = planes[1].getRowStride();
                int chromaRowPadding = chromaRowStride - w / 2;

                int offset = w * h;
                if (chromaRowPadding == 0) {
                    // When the row stride of the chroma channels equals their width, we can copy
                    // the entire channels in one go
                    u_plane.get(yuv_bytes, offset, w * h / 4);
                    offset += w * h / 4;
                    v_plane.get(yuv_bytes, offset, w * h / 4);
                } else {
                    // When not equal, we need to copy the channels row by row
                    for (int i = 0; i < h / 2; i++) {
                        u_plane.get(yuv_bytes, offset, w / 2);
                        offset += w / 2;
                        if (i < h / 2 - 1) {
                            u_plane.position(u_plane.position() + chromaRowPadding);
                        }
                    }
                    for (int i = 0; i < h / 2; i++) {
                        v_plane.get(yuv_bytes, offset, w / 2);
                        offset += w / 2;
                        if (i < h / 2 - 1) {
                            v_plane.position(v_plane.position() + chromaRowPadding);
                        }
                    }
                }

                Mat yuv_mat = new Mat(h + h / 2, w, CvType.CV_8UC1);
                yuv_mat.put(0, 0, yuv_bytes);
                Imgproc.cvtColor(yuv_mat, mBgr, Imgproc.COLOR_YUV2BGR_I420, 3);

                if (flipImage) {
                    Core.flip(mBgr, flipmBgrMat, 1);
                }
            }
        }
    }

@kafan1986 can you confirm the regular rgba() method works correctly?

How did you initialize the mBgr Mat ?

Please collect this information from your case:

  • Image object values: .getFormat(), .getWidth(), .getHeight()
  • from attached Image.Plane objects (for each plane): .getPixelStride(), .getRowStride(), .getBuffer().isDirect()

relates #14342 /cc @Victorlouisdg @floe

Here are the values that was asked:

ImageImage object values: getFormat(): 35 , getWidth(): 2560 , getHeight(): 960
planes[0] : getPixelStride(): 1 , getRowStride(): 2560 , getBuffer().isDirect(): true
planes[1] : getPixelStride(): 2 , getRowStride(): 2560 , getBuffer().isDirect(): true
planes[2] : getPixelStride(): 2 , getRowStride(): 2560 , getBuffer().isDirect(): true