25x performance difference in 16-bit vs 8-bit codes in IVFPQ

[anfatima]

Keeping everything the same and just switching from 8-bit codes to 16-bit codes increases the time to build and train the index by more than 25x. The index contains around 25 million vectors. I only use GPU to train K-Means and train the IVFPQ on an Intel i9 processor 14900KS using 22 threads. Do you have any idea why there is such a significant difference in runtime?

Here is the config I am using,

{
"batch_size": 128,
"use_gpu": true,
"verbose": true,
"num_kmeans_iters": 50,
"num_centroids": 2000,
"num_bits": 16,
"num_quantizers": 16,
"skip_codebook_tables": false,
"use_precomputed_table": true
}

[anfatima]

Just in case, my server has 128 GB DDR5 RAM and 256 GB on NVME SSD. Both 8-bit and 16-bit indices fit in RAM.

[mengdilin]

Can you produce the code snippet you ran for repro?

[alexanderguzhva]

@anfatima 16-bit codes is an enormous overhead indeed. MUCH slower is expected, bcz 8-bit codebook fits in all kinds of caches, plus I expect that there are special dedicated code branches cases for 8-bit codebook,

[github-actions]

This issue is stale because it has been open for 7 days with no activity.