pinia-colada: `useInifiniteQuery()` for infinite scrolling

I’m trying to figure out a flexible API for useInfiniteQuery(). Goals are:

  • Allow to integrate with common practices (Elastic Search, GraphQL, others?)
  • Handle page based navigation
  • Handle cursor based navigation

Currently, I have a very simple version that lacks many features:

useInfiniteQuery() (click to expand source code)
export interface UseInfiniteQueryOptions<
  TResult,
  TError,
  TDataInitial extends TResult | undefined = TResult | undefined,
  TPages = unknown,
> extends Omit<UseQueryOptions<TResult, TError, TDataInitial>, 'query'> {
  /**
   * The function that will be called to fetch the data. It **must** be async.
   */
  query: (pages: NoInfer<TPages>, context: UseQueryFnContext) => Promise<TResult>
  initialPage: TPages | (() => TPages)
  merge: (result: NoInfer<TPages>, current: NoInfer<TResult>) => NoInfer<TPages>
}

export function useInfiniteQuery<TResult, TError = ErrorDefault, TPage = unknown>(
  options: UseInfiniteQueryOptions<TResult, TError, never, TPage>,
) {
  let pages: TPage = toValue(options.initialPage)

  const { refetch, refresh, ...query } = useQuery<TPage, TError, never>({
    ...options,
    // since we hijack the query function and augment the data, we cannot refetch the data
    // like usual
    staleTime: Infinity,
    async query(context) {
      const data: TResult = await options.query(pages, context)
      return (pages = options.merge(pages, data))
    },
  })

  return {
    ...query,
    loadMore: () => refetch(),
  }
}

It’s lacking:

  • per page caching: this means no invalidation per page, no way to refetch everything again)
  • bi directional navigation: this version only has a loadMore() that works with infinite scrolling

By allowing a low level merge function, we can fully control the format of the data and we can end up with just a collection that we iterate on the client. We need initialPage to make types inferrable in merge since it’s both the parameter and return type. Here is full example of it’s usage with scroll:

<script setup lang="ts">
// import { factsApi, type CatFacts } from '@/composables/infinite-query'
import { useInfiniteQuery } from '@pinia/colada'
import { onWatcherCleanup, useTemplateRef, watch } from 'vue'
import { mande } from 'mande'

export interface CatFacts {
  current_page: number
  data: Array<{ fact: string, length: number }>
  first_page_url: string
  from: number
  last_page: number
  last_page_url: string
  links: Array<{
    url: string | null
    label: string
    active: boolean
  }>
  next_page_url: string | null
  path: string
  per_page: number
  prev_page_url: string | null
  to: number
  total: number
}

const factsApi = mande('https://catfact.ninja/facts')

const {
  state: facts,
  loadMore,
  asyncStatus,
  isDelaying,
} = useInfiniteQuery({
  key: ['feed'],
  query: async ({ nextPage }) =>
    nextPage != null ? factsApi.get<CatFacts>({ query: { page: nextPage, limit: 10 } }) : null,
  initialPage: {
    data: new Set<string>(),
    // null for no more pages
    nextPage: 1 as number | null,
  },
  merge(pages, newFacts) {
    // no more pages
    if (!newFacts) return pages
    // ensure we have unique entries even during HMR
    const data = new Set([...pages.data, ...newFacts.data.map((d) => d.fact)])
    return {
      data,
      nextPage: newFacts.next_page_url ? newFacts.current_page + 1 : null,
    }
  },
  // plugins
  retry: 0,
  delay: 0,
})

const loadMoreEl = useTemplateRef('load-more')

watch(loadMoreEl, (el) => {
  if (el) {
    const observer = new IntersectionObserver(
      (entries) => {
        if (entries[0]?.isIntersecting) {
          loadMore()
        }
      },
      {
        rootMargin: '300px',
        threshold: [0],
      },
    )
    observer.observe(el)
    onWatcherCleanup(() => {
      observer.disconnect()
    })
  }
})
</script>

<template>
  <div>
    <button :disabled="asyncStatus === 'loading' || isDelaying" @click="loadMore()">
      Load more (or scroll down)
    </button>
    <template v-if="facts?.data">
      <p>We have loaded {{ facts.data.data.size }} facts</p>
      <details>
        <summary>Show raw</summary>
        <pre>{{ facts }}</pre>
      </details>

      <blockquote v-for="fact in facts.data.data">
        {{ fact }}
      </blockquote>

      <p v-if="facts.data.nextPage" ref="load-more">Loading more...</p>
    </template>
  </div>
</template>

You can run this example locally with pnpm run play and visiting http://localhost:5173/cat-facts


I’m looking for feedback of existing needs in terms of caching and data access to implement a more feature-complete useInfiniteQuery()

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Reactions: 17
  • Comments: 19 (7 by maintainers)

Most upvoted comments

The current implementation works well for basic infinite scroll. However, when I try to add filters, I’m unable to reset the initial state. I tried using a ref outside of useInfiniteQuery() to replace the initial state and reset this object when the filters change, but I ran into issues with the cached query.

Here’s my current implementation:

const filters = reactive({
   search: "",
   tags: [],
})

const pagination = reactive({
   nextPage: 1
})

const query = useInfiniteQuery({
     key: () => ["products", { filters }],
     query: () => api.getProducts(pagination.nextPage, filters);
     merge: () => {
        // handle new page + pagination
     },
     initialPage: {
         products: []
     }
});

watchDeep(filters, () => {
    pagination.nextPage = 1;
    // tried query.loadMore() or queryCache.invalidateQuery(...)
}) 

When the filters change, I want to reset the pagination and update the cache for the query with the new filters.

I’ve seen this use case on booking.com where changing filters resets the infinite scroll and fetches new results.

Is there a way to handle this case ?

I’m trying to add support for useInfiniteQuery in oRPC, and I have a few questions after reviewing your useInfiniteQuery code:

  1. Why does the async query function return pages instead of data?
  2. Why is the merge function’s return type NoInfer<TPages> instead of NoInfer<TResult>?
  3. How do I control fetching the next page?

Looking forward to your insights!

Okay, I just tried this on my machine and it seems to work just fine. If you agree with the principle, I can try to prepare you a PR. The main question is, do you think that this data has its place in the context object ?

Good point, I like your logic much more! And thanks for everything you do for the Vue ecosystem!

@Butch78 doesn’t that work already if you pass a function to key: key: () => ['feed', queryParams]?