Understanding the Performance of Hierarchical Navigable Small World (HNSW) in the Facebook FAISS Library for Indexing Large Databases

Bilal RAZA

Diplomová práce

Abstract:

This thesis explores the performance of the Hierarchical Navigable Small Worlds (HNSW) algorithm within the FAISS library for indexing large databases. As data volumes grow exponentially, Approximate Nearest Neighbor (ANN) algorithms, like HNSW, offer efficient solutions. HNSW constructs a hierarchical graph structure, demonstrating superiority over other ANN algorithms. However, its performance within FAISS, specifically for large databases like Wikipedia, remains underexplored. The research aims to assess HNSW's efficiency in FAISS, focusing on parameters influencing its performance. The results will provide valuable insights into how well the algorithm copes with large datasets and identify potential areas for optimization.

Abstract:

This thesis explores the performance of the Hierarchical Navigable Small Worlds (HNSW) algorithm within the FAISS library for indexing large databases. As data volumes grow exponentially, Approximate Nearest Neighbor (ANN) algorithms, like HNSW, offer efficient solutions. HNSW constructs a hierarchical graph structure, demonstrating superiority over other ANN algorithms. However, its performance within FAISS, specifically for large databases like Wikipedia, remains underexplored. The research aims to assess HNSW's efficiency in FAISS, focusing on parameters influencing its performance. The results will provide valuable insights into how well the algorithm copes with large datasets and identify potential areas for optimization.

Keywords

HNSW, ANN, NLP, FAISS, Wikipedia, Skip List, NSW, Graphs, BERT

Jazyk práce: angličtina

Datum vytvoření / odevzdání či podání práce: 9. 2. 2024

Obhajoba závěrečné práce

• Vedoucí: prof. Dr. Andreas Fischer

JIHOČESKÁ UNIVERZITA V ČESKÝCH BUDĚJOVICÍCH

Přírodovědecká fakulta

Magisterský studijní program / obor:
Artificial Intelligence and Data Science / Artificial Intelligence and Data Science