Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to get (reconstruct) a set of vectors that was associated with a single id? #4183

Open
volkov-maxim opened this issue Feb 10, 2025 · 1 comment

Comments

@volkov-maxim
Copy link

volkov-maxim commented Feb 10, 2025

Hi!
I had associate some sets of vectors with ids, but can't find approach to get back this sets from index.
In case with 2 ids I had got only 2 vectors.

Code for reproduction:
import faiss
import numpy as np

d = 16
k = 2
quantiser = faiss.IndexFlatL2(d)
index = faiss.IndexIVFFlat(quantiser, d, k)
ids = range(1, 3)

index.train(faiss.randn((10, d), 123))
print(index.is_trained)

# Create 1st index.
index.add(faiss.randn((10, d), 345))
index.add(faiss.randn((10, d), 567))
print(index.ntotal)

ref_recons1 = index.reconstruct_n(0, 20)

# Create 2nd index.
index.reset()
index.add_with_ids(faiss.randn((10, d), 345), np.repeat(ids[0], 10))
index.add_with_ids(faiss.randn((10, d), 567), np.repeat(ids[1], 10))

index.set_direct_map_type(faiss.DirectMap.Hashtable)
ref_recons2 = index.reconstruct_n(0, 20)

# Compare items.
for i in range(index.ntotal):
    for j in range(index.ntotal):
        if np.all(ref_recons1[i] == ref_recons2[j]):
            print("i=", i, ", j=", j)

Output:

True
20
i= 9 , j= 1
i= 17 , j= 2

I'm expect 2 sets of vectors:

  • 10 vectors associated with id=1
  • 10 vectors associated with id=2

But, only 2 items are equal and I dont't understand why.
Please help me to implement reconstruction sets of vectors that associated with ids.

@satymish
Copy link
Contributor

Hi @volkov-maxim when I run the code you wrote, I get the following output

True
20
i= 0 , j= 0
i= 3 , j= 3
i= 4 , j= 4
i= 5 , j= 5
i= 6 , j= 6
i= 7 , j= 7
i= 8 , j= 8
i= 9 , j= 1
i= 9 , j= 9
i= 10 , j= 10
i= 11 , j= 11
i= 12 , j= 12
i= 13 , j= 13
i= 14 , j= 14
i= 15 , j= 15
i= 16 , j= 16
i= 17 , j= 2
i= 17 , j= 17
i= 18 , j= 18
i= 19 , j= 19

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants