pnpcore: SPQueryThrottledException while loading document libraries with more than 5000 elements

Im loading the Items by following CamlQuery:

<View> <Query> </Query> <OrderBy Override='TRUE' ><FieldRef Name='ID' Ascending='FALSE' /></OrderBy> <RowLimit>{PAGE_SIZE}</RowLimit> </View> and using list.LoadItemsByCamlQuery(...) with paging as described on API documentation page.

However when a list has more than 5000 elements I’m getting SPQueryThrottledException with message (translated) “The attempted operation is invalid because it exceeds the list view threshold.”

By setting the RowLimit and using pagination I was expecting that im able to work with libraries that contain more than 5000 elements as im not loading them all at once with one server round trip but for each page one.

I found some articles which advice to modify the query so no more than 5000 elements are returned but I thought by setting the row number and using pagination approach only PAGE_SIZE (in my case 500) elements should be returned at once.

Besides that, by using my workaround from #839 when some of the folders contain # or $ im forced to set Caml query scope to RecursiveAll meaning that 5000 elements limit will even be reached faster as im not able to load the items by depth level.

In our solution we are loading all SharePoint securables, therefore we are looking for workaround for this limitation.