You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Mayya Sharipova (Jira)" <ji...@apache.org> on 2021/08/18 16:07:00 UTC
[jira] [Commented] (LUCENE-10054) Handle hierarchy in HNSW graph
[ https://issues.apache.org/jira/browse/LUCENE-10054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17401172#comment-17401172 ]
Mayya Sharipova commented on LUCENE-10054:
------------------------------------------
Proposed .vem index file structure:
+-------------+--------+----------+----------+----------+---------+------+-------+
| FieldNumber | SimFun | VDOffset | VDLength | VIOffset | VILength| dims | docIds
+-------------+--------+----------+----------+----------+---------+------+-------+
--+-------------+--------------+-----+------------+---------------+-----+------------+
| LevelsCount | SizeLevelmax | ... | SizeLevel0 | NodesLevelmax | ... | NodesLevel1
--+-------------+--------------+-----+------------+---------------+-----+------------+
--+----------------------+-----+--------------------+
| graphOffsetsLevelmax | ... | graphOffsetsLevel0 |
--+----------------------+---- +--------------------+
> Handle hierarchy in HNSW graph
> ------------------------------
>
> Key: LUCENE-10054
> URL: https://issues.apache.org/jira/browse/LUCENE-10054
> Project: Lucene - Core
> Issue Type: Task
> Reporter: Mayya Sharipova
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently HNSW graph is represented as a single layer graph.
> We would like to extend it to handle hierarchy as per [discussion|https://issues.apache.org/jira/browse/LUCENE-9004?focusedCommentId=17393216&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17393216].
>
> TODO tasks:
> - add multiple layers in the HnswGraph class
> - modify the format in Lucene90HnswVectorsWriter and Lucene90HnswVectorsReader to handle multiple layers
> - modify graph construction and search algorithm to handle hierarchy
> - run benchmarks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org