pnpcore: Iterating over large folder collection in a paged way still throws ThrottledException
Category
- Bug
Describe the bug
We do have a folder with more than 5000 subfolders. In the documentation, it states that iterating the IFolderCollection using await foreach would take care of paging - but we still run into the ThrottledException immediately. We also tried paging using CAML queries, but even with paging as shown on your documentation, we immediately run into the Exception as well.
using var context = await this.pnpContextFactory.CreateAsync("ConnectionName");
var rootFolder = (await context.Web.Lists.GetByTitleAsync("LibraryName", p => p.RootFolder)).RootFolder;
await foreach (var folder in rootFolder .Folders)
{
// it already throws an exception in the await foreach
}
Steps to reproduce
- Have a document library with more than 5000 folders in the root folder
- Try the above code
- Observe the throttled exception
Expected behavior
I expect the await foreach to do the paging internally as stated in the documentation.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 23 (12 by maintainers)
It might still be a good idea, memory wise, to load page after page and iterate on the page, then clear the page and then load the next page, as page 2-n might be quite huge to store in memory.