Provide generic PaginatedReport Webdav REPORT

[PVince81]

Let's consider this fancy idea: implement a generic PaginatedPropfindReport that works on any collection in any endpoint inside the Sabre tree.

What the report does:

• input as xml
  • list of propfind properties to retrieve
  • offset + limit
• output
  • propfind multi-status response

To compute the result, the report does the following:

1. Call $node = getNodeForPath() on the input URL (ex: "files/admin/sub/dir" or "systemtags/$fileId/", etc)
2. Call $node->getChildren() and cache the result somewhere for this specific path/endpoint
3. Cut out the results out of the result array
4. Format it as multistatus response

The advantage is that this would work for any endpoint.

The drawback is that it is a bit inefficient because it retrieves all results from the sub-node. Now if we can cache those we might be fine.

One idea to improve this would be to create a new interface IPaginatedCollection extends ICollection that provides this method: getChildrenPaginated($offset, $limit)
So if the collection implements that method, the report calls it directly.
If not, then it falls back to use getChildren() + manually cutting out the result.

Now this only applies for a flat list of results which are the children.
The nice thing is that it removes the need to implement pagination for any other endpoint.

By extension, if we want to add filters we could add yet another interface IFilteredCollection with getChildrenFiltered(...) to filter the flat list of results.

@DeepDiver1975 @evert what do you think ?

[PVince81]

Ah, and I guess the pagination thing would only work properly for Depth: 1 because getChildrenPaginated() only applies to direct children. Well, we could also make it be getRecursiveChildrenPaginated() then...

[evert]

A few thoughts:

1. I know people dealing with lengthy calendars/addressbooks sometimes use the sync-collection mechansism to enable paging. This actually works for client that support sync-collection correctly.
2. I think the only good reason to build this feature is if a client does not care about the full result-set. If the client must have the full result-set but wants to split it up into multiple HTTP requests, I think I would try to investigate why that's needed and fix the underlying problem, and not introduce paging.
3. For API's I've always been a fan of never letting the client determine the page-size.
4. If I fetch page 1 and page 2, but in between a change happens, how do you intend to handle that? Seems like that might be very important for robust sync. Using sync-collection does solve this.

[PVince81]

The main purpose for pagination here was to be able to paginate on the web UI and mobile clients to save some bandwidth and improve perf. Not sure how using sync-collection there would help.

Also I guess that sync-collection wouldn't necessarily return the results in the desired sort order, so at some point the clients would need to sync down all results before being able to display the first page of results.

[evert]

@PVince81 yea, you are right. But this falls for me under "a client that does not care about the full result-set" which I think is a legit reason to build this.

Point 3 and 4 still stand though.

[evert]

Actually 4 is now also less important =)

[guruz]

The main purpose for pagination here was to be able to paginate on the web UI and mobile clients to save some bandwidth and improve perf.

I have actually the opposite opinion: It's better to send everything in one request so you have the per-request overhead (and the bad latency) only once.

You could parse the retrieved XML as it comes in if you're worried that you need to wait for the full XML to be there.

[PVince81]

This is not really about waiting for the full XML to be there. It's mostly for mobile clients which might have limited bandwidth (cell network) so instead of retrieving 1000 items at once but only show 20 in the UI they can decide to fetch less in a paginated manner.

[guruz]

Especially on a cell network you're supposed to avoid requests!

Apple says: https://developer.apple.com/library/content/documentation/Performance/Conceptual/EnergyGuide-iOS/DeferNetworking.html#//apple_ref/doc/uid/TP40015243-CH18-SW1 "Batch Transactions"

Android says: https://developer.android.com/training/efficient-downloads/efficient-network-access.html Figure 2. Relative wireless radio power use for bundled versus unbundled transfers.
and Batch Transfers and Connections

[PVince81]

@guruz well as a mobile client you can still choose to retrieve all the results with a PROPFIND instead of REPORT. Or do a REPORT with a big value like 100 or 1000... And if the user really needs to scroll past the 1000 then load the rest afterwards.

[ownclouders]

Hey, this issue has been closed because the label status/STALE is set and there were no updates for 7 days. Feel free to reopen this issue if you deem it appropriate.

(This is an automated comment from GitMate.io.

[PVince81]

I remember @DeepDiver1975 not being happy about a too generic solution. I anyway decided to go with a less generic route with #13915

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.