HyperHDR: Sound visualization freezing after one frame

Steps to reproduce

Enable Audio Capture Card in Effects

Try out Music visualization in Remote Control

Capture Card: https://www.easycoolav.com/products/usb30-video-capture

What is expected?

Music Visualization Overlay on Video Preview

What is actually happening?

Noticed one frame with audio reaction before visualization itself freezes

System

HyperHDR Server: 
- Build:           (HEAD detached at 8d314bc) (Awawa-5b8ed09/8d314bc-1654356765)
- Build time:      Jun  4 2022 15:37:08
- Git Remote:      https://github.com/awawa-dev/HyperHDR
- Version:         18.0.0.0beta3
- UI Lang:         auto (BrowserLang: en-US)
- UI Access:       default
- Avail Capt:      Linux (V4L2)
- Database:        read/write

HyperHDR Server OS: 
- Distribution:   Debian GNU/Linux bookworm/sid
- Architecture:   x86_64
- CPU Model:      AMD Ryzen 5 3400G with Radeon Vega Graphics
- CPU Type:       24
- Kernel:         linux (5.18.0-1-amd64 (WS: 64))
- Qt Version:     5.15.2
- Browser:        Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:103.0) Gecko/20100101 Firefox/103.0 
-

Logs

-- truncated
2022-06-30T13:33:10.572Z [HYPERHDR] Sound device is starting
2022-06-30T13:33:10.572Z [HYPERHDR] Opening device: default:CARD=Capture
2022-06-30T13:33:10.573Z [HYPERHDR] Sound period size = 2048
2022-06-30T13:33:10.574Z [HYPERHDR] Sound buffer size = 4096
2022-06-30T13:33:10.591Z [HYPERHDR] Sound device has started
2022-06-30T13:33:10.591Z [EFFECT0(Music:...)] Begin playing the effect with priority: 1
2022-06-30T13:33:10.691Z [HYPERHDR] Sound stream:  succesfully captured audio data and the sound is detected.
2022-06-30T13:33:10.693Z [MUXER0] Priority 1 is now active
2022-06-30T13:33:10.693Z [MUXER0] Set visible priority to 1
2022-06-30T13:33:10.693Z [IMAGETOLED0] Total index number is: 288 (memory: 288). User sparse processing is: enabled, image size: 80 x 45, area number: 106
2022-06-30T13:33:10.693Z [HYPERHDR0] New priority[1], previous [240]
2022-06-30T13:33:10.694Z [SMOOTHING0] Clearing queued colors before: enabling. Smoothing configuration changed: restarting timer.
2022-06-30T13:33:10.694Z [SMOOTHING0] Smoothing queue is cleared
2022-06-30T13:33:10.694Z [SMOOTHING0] Selecting config (30) => type: Linear, dirMode: true, pause: false, settlingTime: 200ms, interval: 40ms (25Hz), antiFlickTres: 0, antiFlickStep: 0, antiFlickTime: 0
2022-06-30T13:33:10.694Z [SMOOTHING0] Clearing queued colors before: disabling
2022-06-30T13:33:10.694Z [SMOOTHING0] Smoothing queue is cleared
2022-06-30T13:33:20.573Z [EFFECT0(Music:...)] The effect quits with priority: 1
2022-06-30T13:33:20.573Z [EFFECT0(Music:...)] Releasing sound handle 7 for effect named: 'Music: equalizer test (turn on video preview)'
2022-06-30T13:33:20.573Z [HYPERHDR] Sound device is stopping
2022-06-30T13:33:20.573Z [HYPERHDR] Disconnecting from sound driver: 'default:CARD=Capture | FHD Capture, USB Audio
Default Audio Device | Input'
2022-06-30T13:33:25.678Z [HYPERHDR] Sound device has stopped
2022-06-30T13:33:20.573Z [MUXER0] Removed source priority 1
2022-06-30T13:33:20.574Z [MUXER0] Set visible priority to 240
2022-06-30T13:33:20.574Z [IMAGETOLED0] Total index number is: 16068 (memory: 16068). User sparse processing is: enabled, image size: 640 x 360, area number: 106
2022-06-30T13:33:20.574Z [HYPERHDR0] New priority[240], previous [1]
2022-06-30T13:33:20.574Z [EFFECTENGINE0] Effect 'Music: equalizer test (turn on video preview)' has finished.
2022-06-30T13:33:20.574Z [EFFECT0(Music:...)] Deleting effect named: 'Music: equalizer test (turn on video preview)'
2022-06-30T13:33:20.574Z [EFFECT0(Music:...)] Effect named: 'Music: equalizer test (turn on video preview)' is deleted
2022-06-30T13:33:20.574Z [SMOOTHING0] Clearing queued colors before: enabling. Smoothing configuration changed: restarting timer.
2022-06-30T13:33:20.574Z [SMOOTHING0] Smoothing queue is cleared
2022-06-30T13:33:20.575Z [SMOOTHING0] Selecting config (0) => type: Alternative, dirMode: false, pause: false, settlingTime: 150ms, interval: 16ms (62Hz), antiFlickTres: 0, antiFlickStep: 0, antiFlickTime: 0
2022-06-30T13:33:20.575Z [SMOOTHING0] Using alternative smoothing input (0)
2022-06-30T13:33:20.591Z [SMOOTHING0] Using alternative smoothing procedure (0)

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 26 (14 by maintainers)

Most upvoted comments

The main obstacle is FFT: if you didn’t change int16_t sec[SOUNDCAP_RESULT_RES] = {…} it incorrectly (well, at least differently that in the original version) assigns frequencies to the bucket. Also constant LUT table for sinus must be changed to dynamic as it’s still fixed for processing 22050 https://github.com/awawa-dev/HyperHDR/blob/459aa9b81fe04bfd7a259d761a0fdff94178e10d/sources/base/SoundCapture.cpp#L777

awawa-dev on Jul 2, 2022

Yes, ezcoo support is nice and they provide neccesery updates for their devices.

And for migration to 48000. SoundCapLinux.h: SOUNDCAPLINUX_BUF_LENP increase by one to 11, SoundCapLinux.cpp unsigned int exactRate = 22050; to 48000, change else if (exactRate != 22050) to 48000

in SoundCapture.cpp change int16_t resolutionP = 10; to 11 but buckets wont be accurate for 48khz, since we are not limited to 11025 but to 24000 for FFT. Anyway it’s pure for the visual effect not strict pharmacy rules, you can try to multiply each element of sec table by 2 I think. Test it with online freqency generators like this. Good luck.

int16_t sec[SOUNDCAP_RESULT_RES] = {
				2 * 2,   // 0-86
				4 * 2,   // 86-258
				6 * 2,   // 258-516
				12 * 2,  // 516-1032
				24 * 2,  // 1032-2064
				48 * 2,  // 2064-4128
				48 * 2,  // 4128-6192
				512    // 6192-11025
	};

EDIT: Oh, SOUNDCAP_N_WAVE to 2048, SOUNDCAP_LOG2_N_WAVE to 11 in SoundCapture.h But probably new sin LUT table (_lutSin) must be computed…

awawa-dev on Jun 30, 2022