godot: SamplePlayer latency - it's just bad performance.

RtAudio is quite annoying under linux though (at least for me). Messes up my sound all the time and fails to initialize a lot. There is a reason that it is not included in the build on linux by default… PulseAudio and ALSA work way better.

However, I will take another look into the command queue which I mentioned above. Maybe the time between issuing the play command and it getting processed could be shortened as well.

EDIT: Looks like there isn’t much to be improved there. The delay there depends on the hardware latency since the command processing function is called everytime the driver pulls more samples. So moving the start of the playback directly to the voice_play function wouldn’t do anything. So it looks like we are done now. I did what I could, so I hope it is now fixed for everyone.

So I am very annoyed by this problem and messed around with it today. I measured the times at some points by printing out the result from OS::get_ticks_msec. By doing this, I found out that there are two bottlenecks:

1. The time that passes between the call of AudioServer::voice_play() (which happens almost immediately after receiving the triggering _input event) and the queued CMD_PLAY command getting processed within AudioServerSW::driver_process_chunk() is about 10-50 ms. Which is the less significant amount of the whole 200-300 ms delay.

2. After tracking it down to the lowest level, where the driver is being handled, I came to the comclusion that the problem must be there. And I was right.

I am using Linux with the PulseAudio driver. After verifying that AudioDriverPulseAudio is actually being used, I played around with that file a bit, finding out that there is function which returns the latency from PulseAudio. Printing it out showed a latency as high as like 200 ms, just as big as expected. Reading about the PulseAudio API, I played around with the initialization parameters, especially with the buffering attributes pa_buffer_attr, which currently are set to NULL for default initialization. I then set the pa_buffer_attr::tlength to a low value, compiled, and… The delay was gone! So the problem is a too large set buffer size. On the project page of PulseAudio (http://0pointer.de/lennart/projects/pulseaudio/doxygen/structpa__buffer__attr.html) it says that

The server tries to assure that at least tlength bytes are always available in the per-stream server-side playback buffer. It is recommended to set this to (uint32_t) -1, which will initialize this to a value that is deemed sensible by the server. However, this value will default to something like 2s, i.e. for applications that have specific latency requirements this value should be set to the maximum latency that the application can deal with. […]

So it seems that the server sets that to a very high value by default, which causes the high latency. Using the global settings value “audio/output_latency” for the tlength setting (which is being loaded in that file anyway), I managed to fix this problem within a few lines of code. The buffer size does now depend on this setting, which is by default 25 ms (which is totally fine).

So I made a pull request, but also decided to put it up here for discussion. While I was able to fix this, I didn’t really know what I was doing (which is probably normal when you are trying to fix bugs in a huge unkown code base). And it would fix the problem only for pulseaudio. I will take a look into ALSA and maybe the windows driver too later, but I definitely can’t look into the iOS, Android and Mac drivers (since I can’t test it). It is a bit weird anyway that this problem occurs with all of the audio drivers which are independently form each other. So what do you think about this?

Edit: pull request is #7425

Okay, so I dug around a bit, and the audio driver buffer size is based on audio/output_latency in the project settings. By default it’ll be 25ms @ 44100Hz, giving 1102.5 buffer size but because it’s rounded to the next power of two, it’ll actually be a 2048 size buffer or 46.4ms latency. What about the audio mixer? Same principle with audio/mixer_latency, but this time 10ms is actually 11.5ms. Not too bad.

All in all, buffer size alone, you’re talking about 57.9ms latency.

Now comes the interesting part. In main/main.cpp I saw that the main thread operates by processing Input -> Rendering -> Audio Mixer -> Script. That means, from mouse click to actual output, it has to go Input -> [Rendering] -> [Audio] -> Script -> [Input] -> [Rendering] -> Audio. Two whole frames. At say, 60fps, that’s 33ms. At the captured video 30fps it’s worse, at 66ms. EDIT: Digging a bit more and maybe I might be wrong here, but it should at least be 16-33ms.

Now, because the audio driver is updated in a separate thread you’ll get a vsync equivalent, giving an overall total of 33 + 58 + (+46.4 / -0) = 114.2 ± 23.2 ms.

So yeah, quite a bit of wastage that can be shaved off with some tweaking.

The other 100ms I’d say is down to OS latency in the audio system (~10-50ms unless you have a realtime kernel patch), input (wireless mouse?) and stuff I can’t account for without actually touching the code, which I might get around to at some point.

Wow, I did not create this issue to make havoc nor bash devs 😃 I would like to thank you all for confirming my finds. I also assume reading the comments, that the issue is relevant for other Godot users.

@SuperUserNameMan When such latency may not interfere with your gameplay in some different genre of games, it will be noticeable or even ruining the experience in others, therefore if we want Godot to be more versatile game engine, the issue should be resolved as much as possible.

I am sorry, but I have to say, that you had no idea what you were talking about when you described brain anticipating the larger delay. It’s exactly opposite. Brains know very well where things should be in time and space and latency even as small as 25 ms is noticed in the sound anticipating cognitive processes. If two sounds will be played with such delay apart, your brain will clearly be able to recognize them as two separate audio events. Latency below that (12 ms) gives just a feeling, that can’t be described as separate audio events, but brains “feel” that something is not right. So, brain compensating for bigger delays can only be described as less and less comfortable task, that probably ends with completely ruining the game. How ofter did you hear online FPS players with ping 250 ms that were angry and swearing after being killed or not able to headshot enemies because of 250 ms game delay?

Maybe itself it’s not yet an issue worth of a gigantic red neon sign saying that “Godot sucks” (because it clearly doesn’t), but Godot’s audio inefficiencies are evident, as I was trying to state by posting this issue as well as #6963 and #6964

@GodotIsAMess I am glad you see the latency problem as well. It does not make me standing alone here. However I would constrain myself from blaming Godot devs for your fault of not taking care of choosing the right tool for creating your game. If someone spends building something 6 months with the tool he did not check, it’s his own fault if the tool is not the right for the job. If you say that it’s not fair for people who have spent 6 + months only to finally realize about this audio bug, I would say, that Godot’s dev’s will be glad to fully refund your Godot purchase. 😉

@bojidar-bg about your list of possible workarounds.

• ad 1. wav files lag as well.
• ad 2. does actually nothing. May work in space shooters, but not in myriads of other games and sound solutions needed in games. Also who knows what would be the performance cost.
• ad 3. Before posting this issue I tested as much as I could, learning Godot and looking for solutions… since Godot version 1.0.
• ad 4. I wish I would be able to contribute on such lvl. I even started to learn C++ at some point, but I am afraid I am too slow or C++ is too hard… at least for now. But you might be true. I might need to stop reading books and do exercises and jump into code from github, trying to understand and fix something real.
• ad 5. Not everything can be faked. The lack of proper timely response between player’s action and game’s reaction in sound or visual feedback is really easy to spot by an average human being. And some games just need or even are created around obligatory fast sound response. BTW early game consoles and 8-bit computers had better latency than 250 ms 😉

@volzhs they will notice the delay, unless the game will have a strange, visually misleading mechanics or something else, that would be able visually justify the reaction latency. It’s possible, but needs creativity in game design 😃

@punto- @bojidar-bg I am sorry to say this, but no Godot developer should expect Godot users to learn C++ and help with the engine problems etc… After all GDScript was created to be simpler and easier to learn. It’s logical, that Godot will attract many users who were not able to learn other, more complicated and difficult languages, frameworks, game engines. Telling them that they should have the conversation about problems in their pull request thread is not the way to go imho.

The speed of sound in real life, it’s 340 m/s. so, we can hear 250ms delayed sound if something happens about 85 meters out. Can we get this reality with new audio engine? 😃

Just tested it myself, using my phone as a mic. For control, my old laptop has a 400ms (!! it’s old though) delay. Reducing the audio/output_latency to 20ms reduced it to 350ms. Seems like that variable isn’t working 1-to-1 to actual latency. Possibly a race condition somewhere?

EDIT: Arrrgghhh, I realised I forgot to set click-on-press. No, and with repeat testing it seems to only be shaving off 5-10ms

If you think you can fix this then let’s have this conversation in your pull request thread, not here.

On 28 November 2016 at 21:50, GodotIsAMess notifications@github.com wrote:

@SuperUserNameMan https://github.com/SuperUserNameMan Well, I was just saying that because I don’t think it’s fair for people who have spent 6 + months (like myself) only to finally realize about this audio bug and now it’s preventing me from publishing the game. Which is very big. It’s really depressing to be honest, but it’s the risk you take for using open source projects I guess. I just don’t understand how the main developers here don’t view it as a huge problem but will put off fixing it for Q1-Q2 of next year. That makes absolutely no sense to me, but I guess I am the special snowflake here and it only affects my project only. (/s)

Obviously a neon sign on the front page would be dumb, but just saying that’s how I feel personally. Even I were to fix the bug, akien wouldn’t even merge it because he’s so delusionally picky about whitespaces. I would spend more time fixing the damn whitespace in code than actually fixing the bug in C/C++ code.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/6965#issuecomment-263442758, or mute the thread https://github.com/notifications/unsubscribe-auth/AGVmPV5UnxXiAf7-Ln3gEhRyMJ10oE4-ks5rC3a6gaJpZM4KkJRU .

Here are some possible workarounds, for those feeling adventurous and not willing to wait till this is fixed:

1. Have absolutely no delay on the .wav sample, since many samples have. (probably you already tried it tho)
2. Have multiple SamplePlayers, and play them every 125ms apart. If no sound is actually needed, stop them, and hope it doesn’t degrade the game. (assuming that stopping a sounds takes <70ms, and would go undetected)
3. Test many, many scenarios, until you find just one in which some information reaches the AudioServer and is applied in less than 250ms. For example, if it turns out that changing the volume of something is applied in 50ms, then, run multiple repeating samples of the impact, and tune up one of them when the impact happens, then tune it back down before it starts repeating (-> 50ms sound lag). Or, if pausing then continuing the play of some sample (if it’s possible though) takes 100ms to restart the sample, it might be usable for this. And so on…
4. Fork/clone Godot, then start messing around the AudioServer, until you find the real cause of this, then fix it and (maybe) make the whole community happier. In the process, you are also going to gain some nice C++ and bugfixing experience. 😄
5. Don’t add things that require that fast sound response, or fake it somehow. Kind of like early game consoles – if you can’t make real 3D, either fake the 3D or go 2D.

Good job!

To everybody on this issue – if you want the bugfix to get through faster, it would be pretty cool if you can test the #7425 PR, especially on non-linux platforms (but linux testing is also welcome 😃 ). @freemanfromgodotengine @GodotIsAMess @SuperUserNameMan @beamer159 (and others I didn’t spam… yet)

@punto- @bojidar-bg I am sorry to say this, but no Godot developer should expect Godot users to learn C++ and help with the engine problems etc… […] Telling them that they should have the conversation about problems in their pull request thread is not the way to go imho.

( for the sake of minding others’ business and of subtly making the unobvious more obvious, i just would like to point out that the GodotIsAMess’ message to which Punto replied by mail is not exactly the same as the one currently displayed in github … In the initial version, there was a second paragraph :

Even I were to fix the bug, akien wouldn’t even merge it because he’s so delusionally picky about whitespaces. I would spend more time fixing the damn whitespace in code than actually fixing the bug in C/C++ code.

… )

Given the length of this issue and its half-fixed state, I would propose that some of us test the current master branch on all platforms they can and try to assess whether the SamplePlayer latency is correct or not, then we could open a new issue for the platforms/drivers which are still not behaving properly yet.

Haha, yeah. I have very creative yet oddly productive ways of procrastinating actual game making. XD

Anyways, I found that doing one or two tests wasn’t enough so I recorded multiple tests 😛 (n=8): 25ms: x̄=280ms σ=26ms 20ms: x̄=244ms σ=30ms

Seems like my earlier prediction was right! That said, it’s only a small improvement in the grand scheme of things.

Now, before you go about reducing the output_latency to something ridiculously small, I should warn that if it’s too low you’d get lots of popping sounds during playback if the CPU can’t keep up.

@freemanfromgodotengine I don’t expect anything from anyone, but my advice is that if you want to be a successful game developer of course you need to learn c++, especially when nobody can fix your issue after complaining so loudly about it. Even AAA studios with millions of dollars in budgets prefer to have in house people that can fix their engine issues, instead of depending on a 3rd party to solve their problems. We do our best, and as I said on the other thread I agree that this is a problem, and ideally I’d like to see it fixed, but we can’t be responsible for your game. Especially if he brags about being able to fix it, why are we having this discussion?

On 5 December 2016 at 12:13, Rémi Verschelde notifications@github.com wrote:

Remove above comments that went a bit far from the initial topic (+ edited @SuperUserNameMan https://github.com/SuperUserNameMan’s answer to add the proper quote that makes it more understandable).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/6965#issuecomment-264879367, or mute the thread https://github.com/notifications/unsubscribe-auth/AGVmPSZLGd2X2V1AWKq2QESnVVpCpatCks5rFCoSgaJpZM4KkJRU .

Here is an example project similar to DDR. Use WASD for up, down, left, right. When the player hits an arrow, it plays a sound effect. Ideally, the sound would play immediately when the arrow is hit, but the delay makes this type of game unplayable.

In Tap.gd, look at lines 87 and 119. These lines play the sound effect. Right now 87 is commented out. If you uncomment this line, the sound will play when it is supposed to (without any input from the player)

Tap.zip

As I never noticed this delay as a problem, i’ve made some tests too under Windows 8.1, and compared Godot delay to the delay I’m used to experience in the Left4Dead2 video game.

So I recorded the sound (using a camcorder) near my laptop, then I measured the delay between the sound of the mouse click and the sound emitted by the computer using Audacity.

Under windows 8.1, with a Godot 2.0.4.1 project and default audio settings, I get a delay of approximately 200ms.

With Left4Dead2 into the UI, the delay between the mouse click and the sound of the UI is of around 100ms. In game, the delay between the mouse click and the sound of the gun is of around 250ms.