OK. I've failed to be convinced that much of this conversation is relevant to the article although I'm sure it's valuable in other ways.
I'll just leave it in this thread for those interested in a deep dive into FAISS IVF.
OK. I've failed to be convinced that much of this conversation is relevant to the article although I'm sure it's valuable in other ways.
I'll just leave it in this thread for those interested in a deep dive into FAISS IVF.
@Morgan-coded This thread has gotten quite long. I haven't updated the article with anything. Are there any conclusions / insights out of this discussion that are relevant and would make sense to add to the article?
Would you be able to succinctly summarize what you're trying to say in a single sentence or paragraph? It's hard to follow.
EDIT: I think what you're saying is that mean centering doesn't work with FAISS IVF. Do you think the same holds true for HNSW etc?
@Morgan-coded suggested this update to the article for the passage beginning withtxtai's Faiss backend uses exact search through 5,000 rows to clarify what's being discussed here.
txtai's Faiss backend uses exact search through 5,000 rows and switches to IVF above that threshold. At 5,183 rows on scifact, it selects `IVF132,Flat` with `nprobe=8`. Default IVF reduced NDCG at 10 by 41% for LEMUR-2048 (0.54910 β 0.32346), 34% for MUVERA-2048 (0.36757 β 0.24111), and 25% for MUVERA-10240 (0.50021 β 0.37341).
Raising `nprobe` to 64 recovered LEMUR-2048 to 0.5085, 7.4% below exact while remaining approximate. Tuned against tuned, LEMUR-2048 at 0.5085 still edged MUVERA-10240 at 0.48939 with one-fifth the storage. Collection-mean centering narrowed LEMUR's IVF gap from 41% to 30%, but hurt MUVERA under IVF, so it is not a general substitute for index tuning. These are single-dataset scifact/ColBERTv2 numbers from the same harness as the article's tables. Pin `faiss.components` to `IDMap,Flat` when exact search is practical; otherwise raise `nprobe` rather than accepting the default.
Adding @Morgan-coded given he wrote the article π
There is nothing inherent about it that requires a GPU. Did you run into an issue?
Thank you!
The training dataset is from here: https://hf.co/datasets/mecha-org/linux-command-dataset
@maxiw Excellent work with this dataset. It inspired me to run some further analysis using graph networks and agents. See below. Thank you for putting this dataset together!