redux-toolkit: [question] - invalidate a result in cache when using "merge"

@koukalp yeah, there’s a couple different sub-threads going on in this issue, which is also why I’m confused myself 😃

My first note is that the whole “use merge as an infinite query implementation” thing is, frankly, a hack. And I was the one who came up with it, and I knew it was a hack at the time, and it’s still a hack, and I don’t like it.

I want us to ship a “real” infinite query implementation at some point. Ideally this year. I’ve got another thread that’s open to ask folks like you about specific use cases you would need it to handle, and I’d really appreciate it if you could leave some related thoughts over in that thread:

• https://github.com/reduxjs/redux-toolkit/discussions/3174

That said, it’ll also be at least a couple months before we finally have time to comb through the list of open RTKQ feature requests, do some prioritization and planning, and then actually start to work on them. So realistically I wouldn’t expect us to be able to ship something like that until at least the end of the year, and the merge approach is what we’ve got to live with for now.

For your specific case: I don’t have enough context to understand what the actual technical issue is with “deleting an item” and “tag invalidation not working”. Could you provide an actual repo or CodeSandbox that shows this behavior? Right now there’s some comments in this thread that feel kind of vague describing possible behavior in apps, and I don’t know what folks are actually doing as far as configuration, or what specific behavior you’re seeing at runtime.

Also, what about tag invalidation is not working as expected?

@markerikson - I love the answer 😄

Essentially it answers my question - what I am doing is not really recommended / supported and therefore no surprise it doesn’t work like my imagination would like it to work 👍

I will try to prepare a representative case and that should answer all your questions.

Thanks!

@haleyngonadi - for me I need to delegate this to redux, where I can easily create reducers that either add (“onNextPageLoaded”) items or remove a single item (“onDeleteItem”) from the collection.

It just feels like a missed opportunity, bc

1. I am basically duplicating the rtk-query cache
2. I am not getting any benefits of the rtk-query cache while I add a lot of manual “labor”

@markerikson - it feels the discussion went a bit off. My expectations are:

1. I use rtk-query with caching and merge function to implement infinite scroll to view a list of items
2. An item can be deleted from the list => in that case I want the deleted item to disappear from the cache.
3. My issue is, invalidateTags stops to work the moment I use merge and I don’t see a way to manipulate the cache to remove the (deleted) item

As mentioned to @haleyngonadi above, the solution with using redux and a few reducers is quite straightforward, however then I pretty much opt out the rtk-query cache which feels like a missed opportunity

@haleyngonadi : yes, although there’s two things that would need to change in that snippet:

• return a value rather than mutating cache
• in the case of createEntityAdapter specifically, you’d need to do return postAdapter.getInitialState() (if the goal is to reset the data completely to empty), or return postAdapter.setAll(postAdapter.getInitialState(), response) (to replace the data with the latest response contents)

It seems a lot of people are having issues with this, and I have figured out how to make it work with the Entity Adapter, and it works great!

export const feedAdapter = createEntityAdapter<FeedItemProps, number | string>({
  selectId: item => item.id,
  sortComparer: (a, b) => b.created.localeCompare(a.created),
});

export const feedSelectors = feedAdapter.getSelectors();

export const feedApi = api.injectEndpoints({
  endpoints: builder => ({
    getFeedPosts: builder.query<EntityState<FeedItemProps, string> & {
        next_cursor?: string;
      },
      FeedPostRequest
    >({
      query: params => ({
        url: '/url-to-api',
        method: 'GET',
        params,
      }),
      providesTags: (result, _error, arg) =>
        result
          ? [
              ...feedSelectors.selectAll(result).map(({ id }) => ({
                type: 'Feed' as const,
                id,
              })),
              'Feed',
            ]
          : ['Feed'],
      serializeQueryArgs: ({ endpointName }) => {
        return endpointName;
      },

      transformResponse(response: FeedPostResponse) {
        const { data: items, meta } = response;
        return feedAdapter.addMany(
          feedAdapter.getInitialState({
            next_cursor: meta.cursor.next,
          }),
          items,
        );
      },
      forceRefetch({ currentArg, previousArg }) {
        return currentArg?.cursor !== previousArg?.cursor;
      },

      merge: (currentCache, newItems, otherArgs) => {
        const cacheIDs = feedSelectors.selectIds(currentCache);
        const newItemsIDs = feedSelectors.selectIds(newItems);

        const updateIDs = cacheIDs.filter(x => newItemsIDs.includes(x));
        const newIDs = newItemsIDs.filter(x => !cacheIDs.includes(x));
        const deletedIDs = cacheIDs.filter(x => !newItemsIDs.includes(x));

        const updateItems = updateIDs.map(id => ({
          id,
          changes: feedSelectors.selectById(newItems, id) ?? {},
        }));

        const brandNewItems = newIDs
          .map(id => feedSelectors.selectById(newItems, id))
          .reduce(
            (prev, value) => ({ ...prev, [value?.id ?? '-1']: value }),
            {},
          );

        feedAdapter.updateMany(currentCache, updateItems);

        if (otherArgs.arg?.last_seen_post_created_at) {
          feedAdapter.addMany(currentCache, brandNewItems);
        } else {
          const oldState = feedSelectors.selectAll(currentCache);
          feedAdapter.removeAll(currentCache);
          feedAdapter.addMany(currentCache, brandNewItems);
          // remove any posts that exists in deletedIDs
          const updatedState = // remove deletedIDs from oldState
            oldState.filter(post => !deletedIDs.includes(post.id));
          feedAdapter.addMany(currentCache, updatedState);
        }

        currentCache.next_cursor = newItems.next_cursor;
      },
      // 1 hour in seconds
      keepUnusedDataFor: 1800

  })

  })
});

import { createApi, fetchBaseQuery } from "@reduxjs/toolkit/query/react";

// Define a service using a base URL and expected endpoints
export const pokemonApi = createApi({
  reducerPath: "pokemonApi",
  tagTypes: ["Pokemon"],
  baseQuery: fetchBaseQuery({ baseUrl: "https://pokeapi.co/api/v2/" }),
  endpoints: (builder) => ({
    getPokemonByName: builder.query({
      query: (page) => `pokemon?offset=${page * 20}&limit=20`,
      // Only have one cache entry because the arg always maps to one string
      serializeQueryArgs: ({ endpointName }) => {
        return endpointName;
      },
      // Always merge incoming data to the cache entry
      merge: (currentCache, newItems, { arg }) => {
        console.log(arg);
        if (arg === 0) {
          return newItems;
        }
        currentCache.results.push(...newItems.results);
      },

      // Refetch when the page arg changes
      forceRefetch({ currentArg, previousArg }) {
        return currentArg !== previousArg;
      },
      providesTags: ["Pokemon"],
    }),
  }),
});

// Export hooks for usage in functional components, which are
// auto-generated based on the defined endpoints
export const { useGetPokemonByNameQuery } = pokemonApi;

In react i manually set Page to 0 when needed to clear the cache

@markerikson Haha I completely understand. Thank you for the insight you’ve already provided 😃

There’s a couple tricky bits here:

• merge only gets called when there is already data in the cache entry. In other words, merge will not be called for the first successful request (the cache entry is empty, there isn’t anything to “merge” into). It will be called for later successful responses (we already have data in the cache, there’s more data, now you can “merge” them). This is logical, but also confusing.
• That does mean that currently you would need to use the entity adapter in both transformResponse and in merge, because in the initial request case only transformResponse will be called, and you still need the data to be normalized.

Going back to the question of “how do I replace the existing data?”, for the specific case you’ve got here, you’d want to use postsAdapter.setAll(cache, response). The issue is how does that logic in merge know if it should do the “replace” vs the “update” handling. I don’t have an immediate answer for that, for multiple reasons: I don’t know if we do expose a way to know that into merge atm (ie, some argument that indicates “this was a refetch() response” or something); I don’t know your specific codebase’s needs; and also I’m replying to this while doing real day-job work and only have limited brainpower available atm 😃

Thank you for taking time to explain, @markerikson! I call that in transformResponse, is that the wrong way to do it?

getPosts: builder.query({
      query: queryArgs => ({
        url: 'v1/feed?stream=profile',
        method: 'GET',
      }),
      providesTags: (result, error, page) =>
        result
          ? [
              ...result.ids.map(id => ({ type: 'Feed' as const, id })),
              { type: 'Feed', id: 'PARTIAL-LIST' },
            ]
          : [{ type: 'Feed', id: 'PARTIAL-LIST' }],
      serializeQueryArgs: ({ queryArgs }) => {
        const cacheKey = `v1/feed?stream=profile`;
        return cacheKey;
      },

      transformResponse(response: FeedPostResponse) {
        const { data: items, meta } = response;
        return postAdapter.addMany(
          postAdapter.getInitialState({
            next_cursor: meta.cursor.next,
          }),
          items,
        );
      },
      forceRefetch({ currentArg, previousArg }) {
        // Refetch when the cursor changes (which means we are on a new page).
        return currentArg?.cursor !== previousArg?.cursor;
      },

      merge: (cache, response) => {
        postAdapter.addMany(cache, postSelectors.selectAll(response));
        cache.next_cursor = response.next_cursor;
      },
      // 1 hour in seconds
      keepUnusedDataFor: 1800,
    })

@markerikson Ahh, I see. Thank you. Would that still apply if I use createEntityAdapter?

merge: (cache, response) => {
        postAdapter.addMany(cache, postSelectors.selectAll(response));
      }

@haleyngonadi The merge callback is effectively a standard Immer produce function, which means that you can return a complete value instead of mutating the draft argument. So, you should be able to literally return responseData inside of merge (instead of doing something like cachedData.push(newStuff), and it should replace the existing data in that cache entry.

@markerikson A full refetch to happen. Pretty much go back to what it was when the user originally landed on the view.

Now that you mention it, I am getting back data with the request, but it doesn’t replace the cache. So the question indeed is how to replace the data already there!

I am dealing with a recommendation endpoint (no query args) that gives a different set of results every time it is called so I used the merge function to append the results for every call but when I invalidate the tag for this endpoint the request is made and results are added to the existing list rather than replacing.

can the merge function have the information that the tag is invalidated? then we put a check there