RETRO-pytorch: Error Reconstructing FAISS Index
Hiya! Thanks for making this library out in the open!
I’ve been trying to get your training wrapper working, but when it tries to generate the index, I get the following error:
RuntimeError: Error in virtual void faiss::Index::reconstruct(faiss::Index::idx_t, float*) const at /project/faiss/faiss/Index.cpp:48: reconstruct not implemented for this type of index
To reproduce, you can use this google colab: https://colab.research.google.com/drive/1BcOtBpWBGmXX_tOC7WKcHOa9SukWEKpf?usp=sharing
Any help with this would be greatly appreciated!
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 18 (9 by maintainers)
Commits related to this issue
- Use make_direct_map to fix #15 — committed to ncoop57/RETRO-pytorch by ncoop57 2 years ago
- Fix reconstruction error discussed in #15 — committed to ncoop57/RETRO-pytorch by ncoop57 2 years ago
- Merge pull request #16 from ncoop57/reconstruct Fix reconstruction error discussed in #15 — committed to lucidrains/RETRO-pytorch by lucidrains 2 years ago
anyway here is the official recommendation https://github.com/facebookresearch/faiss/wiki/Comparing-GPU-vs-CPU
but in practice I’ve never seen a case where it’s beneficial to use GPU for faiss
faiss cpu is very fast
I recommend never using faiss gpu
@lucidrains fixed the issue with this PR using the guidance from @rom1504: https://github.com/lucidrains/RETRO-pytorch/pull/16 As he alluded to, we lose the memmaping ability 😦. I did find a few more errors after that in the generator function, but I’ll open a separate issue to discuss it and fixes.
pulling in @rom1504 because he might know 😂