faiss-gpu: index_cpu_to_gpu() hangs (doesn't occur with the conda package)

When I install faiss-gpu via pip, index_cpu_to_gpu() seems to hang forever. For example, this code sample hangs for me:

import faiss

index_flat = faiss.IndexFlatL2(16)
gpu_index_flat = faiss.index_cpu_to_gpu(faiss.StandardGpuResources(), 0, index_flat)

The index_cpu_to_gpu() will hang, spinning a CPU at 100% seemingly forever, mostly in libnvidia-ptxjitcompiler.so.510.54.

I confirmed this on two different machines with different GPUs (A100 and A10G).

I found two workarounds:

Downgrade to faiss==1.5.3. I hit the problem with 1.6.0 or later.
Use the conda package. I don’t observe the issue with conda, even with the latest version of faiss-gpu (1.7.2)

Maybe faiss-gpu wheel isn’t built with CUDA 11 support, and that’s why it doesn’t work with A100 / A10G?

Issue Analytics

State:
Created a year ago
Comments:17 (7 by maintainers)

Top GitHub Comments

1reaction

anubhav562commented, Dec 6, 2022

Hey everyone!

Thanks a lot for this chain! My issue got solved!

Issue: I was not able to use FAISS on the NVIDIA A100. The FAISS index was not getting pushed to the GPU!

For people who want a solution at one place. Please follow these steps below:

As faiss-gpu 1.7.3 is not available on pip. You can download the artifact directory (a directory containing multiple wheel files based on different systems) from here -> https://github.com/kyamagu/faiss-wheels/actions/runs/3487300515 . Go to bottom of the page -> locate artifacts and download it to your machine.
Once you have the artifacts folder, you need to see which file is the most suitable for you system. For example: I used the following file: faiss_gpu-1.7.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl . The filename itself indicates the configurations of a system. In the filename cp38 could mean CPython 3.8 (that is what I think). The artifacts directory also has many wheels for different systems. Locate the one compatible for your machine.
Once you have located your compatible wheel just run the following: python -m pip install /path/to/wheel/faiss_gpu-1.7.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

And you are good to go!

Thanks to @kyamagu for maintaining the repo!

1reaction

igor0commented, Apr 7, 2022

OK, yeah, that seems to work! (Not sure how that’s different from what I was trying previously.)

That wheel seems to fix my trivial repro, so it addresses the issue, at least as far as I can tell. Presumably this particular wheel won’t work for someone who has CUDA 10, but that’s a different problem. That’s why PyTorch versions are so complicated.

Top Results From Across the Web

FAISS-GPU cannot be used with GPU version 30xx

I'm writing Python source code and using Faiss. I can use Faiss on CPU, 20xx GPU eg: ...

faiss-gpu - PyPI

A library for efficient similarity search and clustering of dense vectors.

Faiss Users | # Faiss 1.6.3 is out - Facebook

Faiss 1.6.3 is out! This release contains mainly extensions to existing indexes and fixes. ## What's new? On GPU: * Support alternative distances......

Faiss Gpu - :: Anaconda.org

Faiss is a library for efficient similarity search and clustering of dense vectors. It contains algorithms that search in sets of vectors of...

Introduction to Facebook AI Similarity Search (Faiss) - Pinecone

Now, Faiss not only allows us to build an index and search — but ... So, CUDA-enabled Linux users, type conda install -c...