media: Blocked UI when using large playlists

A simple mediaItems.chunked(500).forEach { controller.addMediaItems(it) } doesn’t seem to work for me. Are we supposed to set up a Player.Listener (on the controller side) and send each chunk only after onTimelineChanged() is called?

Yes, simply calling it multiple times in a loop likely makes no big difference I’d say. You’d need to wait with a timed delay or for the callback as you suggested. This isn’t super easy to achieve of course and only helps with setting new items from your code (data flow 1 in my post above). It doesn’t help with data flow 2 (sending updates to controllers) as this needs to be solved inside our library.

I’m curious, does sending incremental updates help ExoPlayer prepare faster too or is it just for fast MediaController=>MediaSession communication?

Should also help with the player preparation (see “paging” player idea that relies on a similar pattern)

that when I do next or previous multiple times, the memory usage spikes up

Probably caused by the pending tasks kept in memory. This should resolve itself by reducing overall delays (as we then don’t keep a long lists of pending tasks anymore).

@tonihei thanks for the much-needed clarifications!

Blocked UI when using large playlists (pretty much resolved)

Highlighted efficient app setup pattern for large playlists (step (b)), in particular optimizing the calls by only setting mediaId values and avoiding calls by suggesting to incrementally send updates instead of all at once. (see https://github.com/androidx/media/issues/592)

Only setting the mediaId definitely helped reduce UI lag significantly, before this the app was almost unusable with a large number of items. I should mention that once the media items are set and the player is prepared/playing, I’m still getting some slowness on the UI side (not to mention ANRs) but I’m not yet sure about the exact cause (no issues if the media items aren’t set).

I did some tests with ~10000 items and after calling MediaController.setMediaItems(), most of the time MediaSession’s onSetMediaItems() is called within 5-20 seconds (guess it’s acceptable(?) for such playlists).

Could you please provide some guidance on how to send incremental updates using a media controller? A simple mediaItems.chunked(500).forEach { controller.addMediaItems(it) } doesn’t seem to work for me. Are we supposed to set up a Player.Listener (on the controller side) and send each chunk only after onTimelineChanged() is called?

I’m curious, does sending incremental updates help ExoPlayer prepare faster too or is it just for fast MediaController=>MediaSession communication?

Unresponsive/slow player

Hit next/previous a couple of times (doesn’t matter if you do it through player notification, player UI or using android auto)… It >>takes like 30-60 seconds before the player resumes playing again.

This sounds strange, but may be related to a huge backlog of work generated by these actions. We’ll have a look at this case to >see if there is something else going on.

The delay is likely not ExoPlayer related, but in the session/controller communication as said above. We’ll check your reproduction steps though, just in case there is some additional problem here.

When this happens MediaSession => MediaController updates are delayed as well. The Player.Listener callbacks on the MediaSession side are also delayed so it’s probably caused by the player taking too long to prepare with large playlists. I hope this will be fixed by the planned improvements.

Not sure if it’s relevant but I was looking at the memory profiler to reduce memory usage and I noticed that when I do next or previous multiple times, the memory usage spikes up and even leads to an OOM if I keep doing it:

My current workaround is to throttle the seekToNext()/seekToPrevious() calls in a forwarding player but it doesn’t work very reliably since the player is slow in general when the playlist is huge.

A “paging” Player wrapper (deserves to be highlighted)

“paging” Player wrapper implementation that let’s you handle large playlists in the session and UI, but only sets a subset on the actual player. This helps with step © for all types of players (not just ExoPlayer).

That is a great idea. If implemented, I think it will solve most if not all of these problems (other optimizations are nice too).

Thanks for the thorough reply - that’s useful to hear and we definitely know that large playlists continue to be problematic.

Let me summarize some of things we’ve done, discussed and are planning to do.

There are two main data flows to look at:

1. Setting a large number of new items on a MediaController (this is technically tracked by #592). This involves (a) App code to generate these items (b) MediaController to MediaSession Binder communication © Player code to handle and prepare the playlist (e.g. ExoPlayer)
2. Sharing the current playlist with controllers (tracked by this issue, #81). This involves: (d) MediaSession to MediaController Binder communication (e) MediaSession to legacy MediaSessionCompat compatibility layer

Note that all these steps need to handle the large number of items somehow and to improve the performance, there are two options: optimize the operation or avoid the operation as much as possible.

What we’ve done so far

• Hugely optimized the time for (b) and (d) by reducing the Bundle operations required for individual items. (see this comment: https://github.com/androidx/media/issues/81#issuecomment-1378747452)
• Avoided sending updated for (d) when nothing changed.
• Highlighted efficient app setup pattern for large playlists (step (b)), in particular optimizing the calls by only setting mediaId values and avoiding calls by suggesting to incrementally send updates instead of all at once. (see #592)
• Further internal analysis on how the Binder communication influences the performance. Part of this highlighted that local Binder calls in particular are not as optimized as they should be (see my comment here, that is not entirely true in hindsight: https://github.com/androidx/media/issues/81#issuecomment-1671822737)

What we are planning to do next

• Some local binder improvements for (b) and (d). This only helps for controller<->session communication within the same process, but should noticeably improve some of the cases where you use a controller in the same app (that is likely to run in the same process). This will be part of release 1.2.0-rc01.

For later releases:

• A new setting that restricts the number of items shared with legacy session (e) and MediaController instances in other processed (d). This avoids the issue by simply shortening the playlist. This is also a loss of functionality for remote Media3 controllers. For legacy controllers it never worked anyway.
• Send incremental updates to Media3 controllers (d) to re-add the functionality we may remove with the step above.
• a “paging” Player wrapper implementation that let’s you handle large playlists in the session and UI, but only sets a subset on the actual player. This helps with step © for all types of players (not just ExoPlayer).

What we are not planning to do

• Any improvements for (a). This outside of our control and needs to be handled by your app directly. You can use similar ideas of course: load items in batches, generate incremental updates where needed, etc.
• Use a new background thread for any processing. This may sound counter-intuitive, but it really wouldn’t help.
  • Most of the operations that may take time in MediaSession, MediaController and ExoPlayer are already on a background thread anyway.
  • Moving further updates like the Binder communication to a background thread would likely only help for local in-process calls, which are already optimized by the planned steps above. (The remote cross-process calls are already on background threads)
  • Another reason not to do this is that it won’t help with the user-visible delay in an app. Even if you use another background thread to transmit/receive data, the user still needs to wait until this is finished to see the effect in the UI. It also hugely complicates the correctness and ordering of other operations happening around the same time.
  • It’s likely more beneficial to work on reduced updates or incremental updates as explained above, as these can be done quickly and result in a responsive UI.

---

Some comments on your concrete proposals from the comment above:

Hit next/previous a couple of times (doesn’t matter if you do it through player notification, player UI or using android auto)… It takes like 30-60 seconds before the player resumes playing again.

This sounds strange, but may be related to a huge backlog of work generated by these actions. We’ll have a look at this case to see if there is something else going on.

Please make MediaController do all blocking operations on a background thread because the current behavior leads to inevitable crashes and ANRs on many devices

ANRs can be caused by any of the steps mentioned above if they run within one main thread iteration. We’ll submit some improvements, and you can also check in your app code that it is as efficient as possible and not handling too many items at once if it can be avoided. The code in MediaController in particular is not the slow part (and also not blocking). And as explained above, we are also not planning to consider any further background threads because we don’t see a case where they would help further.

Is there any way to customize ExoPlayer so it doesn’t do whatever unnecessary things it’s doing that’s causing this delay during the next/previous/play/pause commands?

The delay is likely not ExoPlayer related, but in the session/controller communication as said above. We’ll check your reproduction steps though, just in case there is some additional problem here.

Original MediaPlayer performance

This comparison is not really useful because this setup neither has a player that supports playlists nor a session that knows and shares all the items. So it’s fast by removing all the functionality and avoiding all the work.

@rohitjoins probably knows this a bit better, but if my understanding is correct, the blocking happens on the reading side (=the MediaController) while it’s waiting for all the data to be transferred. So I think using a different thread for your player on the MediaSession process won’t help?