redux-persist: Android AsyncStorage: "Error restoring state" for large datasets

This library has been working perfectly for me with smaller size state trees, but trying to use it on bigger ones I’m running into these errors when relaunching the app:

10:39:16 redux-persist/getStoredState: Error restoring data for key: entities {}

10:39:18 Possible Unhandled Promise Rejection (id: 0):
Couldn't read row 0, col 0 from CursorWindow.  Make sure the Cursor is initialized correctly before accessing data from it.

Specifically, I’m trying to store a lot of data from a server locally. To give an idea of the size, running JSON.stringify(payload).length gives 2368916, so it looks like it should be within the 6MB limit.

I’m not having any of the performance issues described in #185 either - the app runs fairly smoothly (perhaps because writes are used sparingly). It’s just a case of closing the app and reopening it leading to this error when it tries to rehydrate.

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 15 (4 by maintainers)

Commits related to this issue

Update readme with migration snippet Original issues: https://github.com/rt2zz/redux-persist/issues/199 https://github.com/rt2zz/redux-persist/issues/284 Converastions related to provided snippet: h... — committed to maxkomarychev/redux-persist-filesystem-storage by maxkomarychev 6 years ago
Update readme with migration snippet Original issues: https://github.com/rt2zz/redux-persist/issues/199 https://github.com/rt2zz/redux-persist/issues/284 Converastions related to provided snippet: h... — committed to maxkomarychev/redux-persist-filesystem-storage by maxkomarychev 6 years ago
Update readme with migration snippet Provide sample snippet which lets app migrate data previously stored in `AsyncStorage` to `redux-persist-filesystem-storage` Original issues: https://github.com/... — committed to maxkomarychev/redux-persist-filesystem-storage by maxkomarychev 6 years ago
Update readme with migration snippet Provide sample snippet which lets app migrate data previously stored in `AsyncStorage` to `redux-persist-filesystem-storage` Original issues: https://github.com/... — committed to maxkomarychev/redux-persist-filesystem-storage by maxkomarychev 6 years ago
redux-persist: Add compression, on Android. The motivation that makes this important to do now is the storage limits imposed by RN's AsyncStorage on Android. In particular, on Android the open-sourc... — committed to zulip/zulip-mobile by roberthoenig 6 years ago

Most upvoted comments

@kenma9123 I’ve created a project on npm with a cleaned up version of the above code. Check out https://www.npmjs.com/package/redux-persist-filesystem-storage

+25

robwalkerco on Apr 10, 2017

@kenma9123 I’ve worked around this issue by creating my own storage for Android that uses the filesystem rather that AsyncStorage.

Hopefully that will help others with this same issue. Perhaps there would be a way to integrate the storage implementation back into redux-persist so as to provide a more robust React Native Android storage solution?

Expand to see the code

/**
* @flow
*/

import RNFetchBlob from 'react-native-fetch-blob'

const DocumentDir = RNFetchBlob.fs.dirs.DocumentDir
const storagePath = `${DocumentDir}/persistStore`
const encoding = 'utf8'

const toFileName = (name: string) => name.split(':').join('-')
const fromFileName = (name: string) => name.split('-').join(':')

const pathForKey = (key: string) => `${storagePath}/${toFileName(key)}`

const AndroidFileStorage = {
  setItem: (
    key: string,
    value: string,
    callback?: ?(error: ?Error) => void,
  ) =>
    new Promise((resolve, reject) =>
      RNFetchBlob.fs.writeFile(pathForKey(key), value, encoding)
        .then(() => {
          if (callback) {
            callback()
          }
          resolve()
        })
        .catch(error => {
          if (callback) {
            callback(error && error)
          }
          reject(error)
        })
  ),
  getItem: (
    key: string,
    callback?: ?(error: ?Error, result: ?string) => void
  ) =>
    new Promise((resolve, reject) =>
      RNFetchBlob.fs.readFile(pathForKey(toFileName(key)), encoding)
        .then(data => {
          if (callback) {
            callback(null, data)
          }
          resolve(data)
        })
        .catch(error => {
          if (callback) {
            callback(error)
          }
          reject(error)
        })
  ),
  removeItem: (
    key: string,
    callback?: ?(error: ?Error) => void,
  ) =>
    new Promise((resolve, reject) =>
      RNFetchBlob.fs.unlink(pathForKey(toFileName(key)))
        .then(() => {
          if (callback) {
            callback()
          }
          resolve()
        })
        .catch(error => {
          if (callback) {
            callback(error)
          }
          reject(error)
        })
  ),
  getAllKeys: (
    callback?: ?(error: ?Error, keys: ?Array<string>) => void,
  ) =>
    new Promise((resolve, reject) =>
      RNFetchBlob.fs.exists(storagePath)
      .then(exists =>
        exists ? Promise.resolve() : RNFetchBlob.fs.mkdir(storagePath)
      )
      .then(() =>
        RNFetchBlob.fs.ls(storagePath)
          .then(files => files.map(file => fromFileName(file)))
          .then(files => {
            if (callback) {
              callback(null, files)
            }
            resolve(files)
          })
      )
      .catch(error => {
        if (callback) {
          callback(error)
        }
        reject(error)
      })
  ),
}

export default AndroidFileStorage

+25

robwalkerco on Apr 7, 2017

For me, this solution worked. Add this piece of code at the end of onCreate() in MainApplication.java

try {
  Field field = CursorWindow.class.getDeclaredField("sCursorWindowSize");
  field.setAccessible(true);
  field.set(null, 100 * 1024 * 1024); //100MB
  } catch (Exception e) {
 if (BuildConfig.DEBUG) {
  e.printStackTrace();
  }
  }

Also import these on top of the MainApplication.java

import android.database.CursorWindow;
import java.lang.reflect.Field;

shehzadosama on Jan 12, 2022

I just had the same issue and @robwalkerco 's lib fixed it - thanks a lot man!

(btw I think this issue can be closed…?)

kelset on Sep 11, 2017

I’m having a similar issue. I suspect we are hitting the max sqlite cursor window size on Android which is 2MB: http://stackoverflow.com/questions/21432556/android-java-lang-illegalstateexception-couldnt-read-row-0-col-0-from-cursorw

johnwayner on Apr 3, 2017

@robwalkerco Thanks you saved my day.

rahulbhankar786 on Sep 25, 2019