zephyr: drivers: video: video_stm32_dcmi: Detect and recover from: HAL_DCMI_ErrorCallback
Summary
With several of the different sketches that do things like display a camera image to a display of some form. Like Arduino Gigia Display shield, or some TFT display like an ST7796 or ILI9341, As I mentioned in the issue: #92352 I will receive several messages like:
[00:00:12.307,000] <dbg> video_stm32_dcmi: HAL_DCMI_FrameEventCallback: Failed to get buffer from fifo
Which in that issue I would hope to reuse the oldest frame…
But then I often times then get the message:
[00:00:14.851,000] <wrn> video_stm32_dcmi: HAL_DCMI_ErrorCallback
And the camera completely stops at that point
Describe the solution you’d like
I wish that the callback could correct the issue and recover.
Alternatives
Or wish there was a way for the client code to detect this and call something to recover.
Detecting this might simply be, that I would expect an image to be ready by my timeout… If not then call the recovery…
Additional Context
Most of my testing is done using the ArduinoCore-zephyr setup. Currently running the latest github code. The west update from their points us to a zephyr snaphshot from about a month ago.
About this issue
- Original URL
- State: open
- Created a year ago
- Reactions: 2
- Comments: 36 (29 by maintainers)
I think I am pretty late on this topic and I couldn’t follow all the discussions, apologies for that. In one of those threads talking about HAL_DCMI_ErrorCallback I think I wrote once that it could be due to the sensor sending data too rapidly. There are two potentially limiting factors on the receiver side:
the DCMI interface itself. The kernel clock of the DCMI must be at least 2.5 time faster than the pixel clock received by the sensor. So it is important to see what is the kernel clock frequency which clock the DCMI and dividing this by 2.5 will give the maximum allowed pixel clock frequency it can handle. (in the CSI context we have been discussing about having helpers to tackle those HW capabilities via usage of the link-frequencies from video-interface.yaml but that’s not really in place yet)
the DMA capabilities itself (and related to the downstream memory access). The DCMI IP receive data and store them into its internal FIFO. Then the DMA goes to read those data and store them somewhere else. So the performance of the DMA and memory access can also be a bottleneck.
To cop with those, it is usually possible to limit the sensor pixel clock rate.
As a matter of fact, the STM32L4 (stm32l4r9i_disco) is also facing this issue (HAL_DCMI_ErrorCallback) when used with the st_b_cam_omv shield. I haven’t got time to do that cleanly and post a PR for that, however just for explanatiion / demo it is possible to make it work (and use UVC sample) by simply forcing the OV5640 PLL frequency lower:
That done, OV5640 streaming into STM32L4R9I_DISCO is working just fine.
Thanks,
I should have mentioned, that the overlay I am using has:
And yes the xclk, is the clock signal and running at 12mhz more or less:
Will drill down farther…
Another update: I now have a version that runs on MBED, in particular I have it displaying currently on an ST7796 display. (Display code is slow, did first pass port the zephyr version I did to MBED). I made changes to Arducam library, My version is up at: https://github.com/KurtE/Arducam_dvp/tree/experiments_debug My sketch and ST7706 code are up in my github project: https://github.com/KurtE/Arduino_GIGA-stuff I am currently configured to use SDRam for the frame buffer. So far it has run for several minutes without failing.
Note: There are a lot of differences on how this Camera code works on MBED versus on Zephyr. On Zephyr you have multiple buffers defined, one is handed to the Camera, that is constantly is pounding the SDRAM with each frame. It then process an interrupt when each frame completes, and it then tries to grab a frame buffer from the free list, where it memcpy the data from the camera. The Arduino wrapper in the ArduinoCore-zephyr or my sketch then needs to swap the bytes for each pixel and then the code writes the data out of SPI to the TFT or over the xxx to write to the GigaDisplayShield… So lots of code hitting the SDRAM maybe at the same time…
With the MBED: when I call:
int err = cam.grabFrame(fb);The code does:So it starts the camera in SNAPSHOT mode (one frame), and then waits up to the timeout in a hard loop until the DCMI_CR_CAPTURE bit is cleared, when the hardware detects the end of the frame and stops…
Then the application or camera library if necessary swaps the bytes and then the application does whatever it needs to do, like send the bytes out to the display.
So with the read an image, write to display code, only one thing is touching the SDRAM. And by default if your code does not set the fb memory like: fb.setBuffer(fb_buf); The camera code will allocate the memory using malloc (not SDRAM).
Wondering if the Video/Camera code on Zephyr should have the ability for SnapShot mode?
Maybe the problem with ov5640 is the master clock is not enough ? For CSI2 mode on RT1170-EVK, the clock frequency is 24 MHz. For DVP mode on ST boards, I don’t know … just a guess …
Sorry everyone, was busy… watering garden.
I tried the a version of my zephyr sketch/… Made a copy of the GC2145 sketch… edited overlay and config… It is up at: https://github.com/KurtE/zephyr_test_sketches/tree/master/ov5640_to_ili9341 Set up with Portenta H7…
I am not getting past first base with it. That is I don’t believe that I have the reset and Powerdown pins configured properly…
Currently I have:
I edited the cfg to have
CONFIG_VIDEO_OV5640=yinstead ofCONFIG_VIDEO_GC2145=yThe issue is the I2C communications is NAKing out.
I believe the startup is supposed to look like:
The init code for ov5640 has:
I know with some setups, We have had some luck with defaulting some of these pins to either INPUT or INPUT_PULLUP or INPUT_PULLDOWN… INPUT was nice if the cameras have their own PU/PD resistors to put them in their normal powered up and not reset setate…
Trying to now to setup on a teensy and see what this signals are doing on it…
Avoiding the
memcpy()operation would make most sense, and maybe support for SNAPSHOT mode is the way. Thememcpy()was convenient as a way to get the driver going, but for the way the video APIs work, swapping the buffer would make more sense than copying it…I will follow-up about the
bus-typeon https://github.com/arduino/ArduinoCore-zephyr/issues/152, it is possible that it is one reason for which the OV5640 stays silent.@avolmat-st - @KurtE
As @KurtE mentioned we are both playing around with the different cameras on the Arduino GIGA and the Arduino Portenta H7. While @KurtE has been working directly with the Zephyr examples I have been primarily working with the ArduinoCore-Zephry through the Arduino IDE. And as could be expected we are both seeing the same issues with the Cameras.
Right not I am trying to get the OV5640 working with both the GIGA and the H7 with just about zero luck.
Issue:
The one thing I am curious about is what bus-type is being used for the GIGA and H7 which I asked on the arduinoCore-zephyr: https://github.com/arduino/ArduinoCore-zephyr/issues/152.
With the other camera’s no bus-type is specified so not 100% sure. Note tried both the GIGA and H7 with no bus specified and with bus-type specified as parallel. Neither works.
The other thing I noticed is that you all specify the configuration for the each support framesize. While what we do is something similar to what is done in the GC2145:
where:
If you all have any thoughts it would help.
One would have to compare the speed of the image sensor to be sure, but good to have this as reference!
It will help to have more image sensor ported and more features in the current sensors, to compare across them. A WIP task!
If the SDRAM is not fast enough to push the data from DCMI, this can lead to a DMA error!
And this is not the first STM32 that seems to have this issue: