rodio: span_length is the wrong abstraction

When decoding audio you get chunks with constant sample rate and channel count. It seemed natural to extend that into rodio’s audio pipeline. The consumer of audio gets the guarantee that the sample rate and channel count will not change for the next n-samples. The consumer can then deal with that like this:

let mut resampler = Resampler::new();
loop {
  resampler.set_sample_rate(source.sample_rate());
  for _ in 0..source.current_span_len() {
    resample.resample(source.next()?);
  }
}

This seems very efficient right? It is however no different then:

let mut resampler = Resampler::new();
loop {
  resampler.set_sample_rate(source.sample_rate());
  while !source.parameters_changed() {
    resample.resample(source.next()?);
  }
}

Especially if we let the compiler know that the parameters usually do not change using:

impl Source {
  #[cold]
  fn parameters_changed() -> bool;

  ..
}

Currently at least skipping, seeking and queue are non-trivial in rodio since we may not break our promised span_len. That span len is however never used except to create the boolean parameters_changed via a comparison to an increasing index. Replacing current_span_len by parameters_changed makes rodio code far simpeler. For example seek:

impl Source {
  fn seek(&mut self, some_params: some_type) {
    self.parameters_changed = true
    // and some code to guarantee frame alignment
   //  but nothing for span alignment since spans do not exist anymore!
    ..
  }

  fn parameters_changed() -> bool {
    self.parameters_changed
  }
}

All the complex code in queue can just be removed, it can simply forward parameters_changed until a source ends: then it just switches it to true. Right now it is has to play silence to fill the current span if a source ends early. So this change will not only make the code far far simpler it will improve audio latency!

Regarding breaking the Source API, an argument along the line of https://github.com/RustAudio/rodio/issues/690#issuecomment-2612265055 can be made. This would also replace #690.

I think this is a wonderful fix/improvement. I might be wrong. Looking forward to your input.

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 34 (19 by maintainers)

Most upvoted comments

If we want SIMD, then we should work on buffers, at least smaller ones (say 4 or more frames/samples).

if the iterator chain is properly in-lined we can simply buffer a bit at every mixer. Then we should get SIMD in the chain up till the mixer.

Another thing to think of is that when we check for parameter changes on every frame this may prevent some compiler optimizations.

I do not really think so. Maybe there is some loop unrolling but then that would need to be guarded by a branch because the current_span _length can have any value (as far as the compiler knows). On the other hand compilers are magic so maybe?

In any case if we want better throughput it is possible to use buffers

That would work at the exchange of latency, since we can not abort early.

Rubato says that it supports SIMD

The wip rubato backend I wrote takes a ‘fixed’ (dynamic) buffer size. It incurs some latency, I kept the buffer size to the minimum required (which was still a lot (2k samples I think?))

w.r.t. this discussion of SIMD auto-vectorization, branching, and what not, I thought this article was very interesting: https://matklad.github.io/2023/04/09/can-you-trust-a-compiler-to-optimize-your-code.html.

Key takeaways are that to leverage compiler SIMD auto-magic, we need to:

  1. Process in chunks of fixed or at least predictable size
  2. Within that processing, do not have any branches
  3. Take care to not have short-circuiting evaluations by using bit-wise instead of logical operators: & instead of && etc.

ChannelVolume / ChannelCountConverter / UniformSourceIterator may be hard to implement if there is no guarantee the parameters will not change between each 0…num_channels

They currently already may, but they practically do not change in between frames.

perhaps the parameters should not change until all Samples have been emitted for the current num_channels (e.g. for a 6 channel source the parameters can only change every 6 channels).

It makes no sense to change sample-rate or channel count mid frame. I think that is a fine demand to add to the source API.

related

Do we demand sources only emit whole frames?

Its a balance,

  • if we do that the API becomes stricter, adding new sources will be harder. It also opens up a new source of bugs. emitting a half frame at the end will be noticeable through subtle bugs in other parts of rodio (channels switching around).
  • Leaving the API as is puts all the work on us, though I think only queue needs to handle this right? And it is relatively doable.