You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Mike Sokolov (Jira)" <ji...@apache.org> on 2022/07/26 19:50:00 UTC

[jira] [Commented] (LUCENE-10054) Handle hierarchy in HNSW graph

    [ https://issues.apache.org/jira/browse/LUCENE-10054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571588#comment-17571588 ] 

Mike Sokolov commented on LUCENE-10054:
---------------------------------------

what is it with this issue that spammers love so much!? I wonder if we
could somehow lock it as read-only ...



> Handle hierarchy in HNSW graph
> ------------------------------
>
>                 Key: LUCENE-10054
>                 URL: https://issues.apache.org/jira/browse/LUCENE-10054
>             Project: Lucene - Core
>          Issue Type: Task
>            Reporter: Mayya Sharipova
>            Priority: Major
>              Labels: vector-based-search
>             Fix For: 9.1
>
>          Time Spent: 20h 20m
>  Remaining Estimate: 0h
>
> Currently HNSW graph is represented as a single layer graph. 
>  We would like to extend it to handle hierarchy as per [discussion|https://issues.apache.org/jira/browse/LUCENE-9004?focusedCommentId=17393216&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17393216]. 
>  
> TODO tasks:
> - add multiple layers in the HnswGraph class
>  - modify the format in  Lucene90HnswVectorsWriter and Lucene90HnswVectorsReader to handle multiple layers
> - modify graph construction and search algorithm to handle hierarchy
>  - run benchmarks



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [jira] [Commented] (LUCENE-10054) Handle hierarchy in HNSW graph

Posted by Mayya Sharipova <ma...@elastic.co.INVALID>.
Thanks very much Michael and David for following up on this.
I also asked about this in the ASF slack "#askinfra" channel, but so far
did not get any response.

On Wed, Jul 27, 2022 at 5:14 PM David Smiley <ds...@apache.org> wrote:

> FYI I had filed https://issues.apache.org/jira/browse/INFRA-23503
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Tue, Jul 26, 2022 at 3:54 PM Michael Sokolov <ms...@gmail.com>
> wrote:
>
>> searching JIRA for "slkjfdf" I found a few issues in other projects,
>> but none seems to be getting the same degree of spam love
>>
>> On Tue, Jul 26, 2022 at 3:50 PM Mike Sokolov (Jira) <ji...@apache.org>
>> wrote:
>> >
>> >
>> >     [
>> https://issues.apache.org/jira/browse/LUCENE-10054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571588#comment-17571588
>> ]
>> >
>> > Mike Sokolov commented on LUCENE-10054:
>> > ---------------------------------------
>> >
>> > what is it with this issue that spammers love so much!? I wonder if we
>> > could somehow lock it as read-only ...
>> >
>> >
>> >
>> > > Handle hierarchy in HNSW graph
>> > > ------------------------------
>> > >
>> > >                 Key: LUCENE-10054
>> > >                 URL:
>> https://issues.apache.org/jira/browse/LUCENE-10054
>> > >             Project: Lucene - Core
>> > >          Issue Type: Task
>> > >            Reporter: Mayya Sharipova
>> > >            Priority: Major
>> > >              Labels: vector-based-search
>> > >             Fix For: 9.1
>> > >
>> > >          Time Spent: 20h 20m
>> > >  Remaining Estimate: 0h
>> > >
>> > > Currently HNSW graph is represented as a single layer graph.
>> > >  We would like to extend it to handle hierarchy as per [discussion|
>> https://issues.apache.org/jira/browse/LUCENE-9004?focusedCommentId=17393216&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17393216
>> ].
>> > >
>> > > TODO tasks:
>> > > - add multiple layers in the HnswGraph class
>> > >  - modify the format in  Lucene90HnswVectorsWriter and
>> Lucene90HnswVectorsReader to handle multiple layers
>> > > - modify graph construction and search algorithm to handle hierarchy
>> > >  - run benchmarks
>> >
>> >
>> >
>> > --
>> > This message was sent by Atlassian Jira
>> > (v8.20.10#820010)
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: issues-help@lucene.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>

Re: [jira] [Commented] (LUCENE-10054) Handle hierarchy in HNSW graph

Posted by Michael Sokolov <ms...@gmail.com>.
Thanks David

On Wed, Jul 27, 2022 at 5:13 PM David Smiley <ds...@apache.org> wrote:
>
> FYI I had filed https://issues.apache.org/jira/browse/INFRA-23503
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Tue, Jul 26, 2022 at 3:54 PM Michael Sokolov <ms...@gmail.com> wrote:
>>
>> searching JIRA for "slkjfdf" I found a few issues in other projects,
>> but none seems to be getting the same degree of spam love
>>
>> On Tue, Jul 26, 2022 at 3:50 PM Mike Sokolov (Jira) <ji...@apache.org> wrote:
>> >
>> >
>> >     [ https://issues.apache.org/jira/browse/LUCENE-10054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571588#comment-17571588 ]
>> >
>> > Mike Sokolov commented on LUCENE-10054:
>> > ---------------------------------------
>> >
>> > what is it with this issue that spammers love so much!? I wonder if we
>> > could somehow lock it as read-only ...
>> >
>> >
>> >
>> > > Handle hierarchy in HNSW graph
>> > > ------------------------------
>> > >
>> > >                 Key: LUCENE-10054
>> > >                 URL: https://issues.apache.org/jira/browse/LUCENE-10054
>> > >             Project: Lucene - Core
>> > >          Issue Type: Task
>> > >            Reporter: Mayya Sharipova
>> > >            Priority: Major
>> > >              Labels: vector-based-search
>> > >             Fix For: 9.1
>> > >
>> > >          Time Spent: 20h 20m
>> > >  Remaining Estimate: 0h
>> > >
>> > > Currently HNSW graph is represented as a single layer graph.
>> > >  We would like to extend it to handle hierarchy as per [discussion|https://issues.apache.org/jira/browse/LUCENE-9004?focusedCommentId=17393216&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17393216].
>> > >
>> > > TODO tasks:
>> > > - add multiple layers in the HnswGraph class
>> > >  - modify the format in  Lucene90HnswVectorsWriter and Lucene90HnswVectorsReader to handle multiple layers
>> > > - modify graph construction and search algorithm to handle hierarchy
>> > >  - run benchmarks
>> >
>> >
>> >
>> > --
>> > This message was sent by Atlassian Jira
>> > (v8.20.10#820010)
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: issues-help@lucene.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [jira] [Commented] (LUCENE-10054) Handle hierarchy in HNSW graph

Posted by David Smiley <ds...@apache.org>.
FYI I had filed https://issues.apache.org/jira/browse/INFRA-23503

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Tue, Jul 26, 2022 at 3:54 PM Michael Sokolov <ms...@gmail.com> wrote:

> searching JIRA for "slkjfdf" I found a few issues in other projects,
> but none seems to be getting the same degree of spam love
>
> On Tue, Jul 26, 2022 at 3:50 PM Mike Sokolov (Jira) <ji...@apache.org>
> wrote:
> >
> >
> >     [
> https://issues.apache.org/jira/browse/LUCENE-10054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571588#comment-17571588
> ]
> >
> > Mike Sokolov commented on LUCENE-10054:
> > ---------------------------------------
> >
> > what is it with this issue that spammers love so much!? I wonder if we
> > could somehow lock it as read-only ...
> >
> >
> >
> > > Handle hierarchy in HNSW graph
> > > ------------------------------
> > >
> > >                 Key: LUCENE-10054
> > >                 URL:
> https://issues.apache.org/jira/browse/LUCENE-10054
> > >             Project: Lucene - Core
> > >          Issue Type: Task
> > >            Reporter: Mayya Sharipova
> > >            Priority: Major
> > >              Labels: vector-based-search
> > >             Fix For: 9.1
> > >
> > >          Time Spent: 20h 20m
> > >  Remaining Estimate: 0h
> > >
> > > Currently HNSW graph is represented as a single layer graph.
> > >  We would like to extend it to handle hierarchy as per [discussion|
> https://issues.apache.org/jira/browse/LUCENE-9004?focusedCommentId=17393216&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17393216
> ].
> > >
> > > TODO tasks:
> > > - add multiple layers in the HnswGraph class
> > >  - modify the format in  Lucene90HnswVectorsWriter and
> Lucene90HnswVectorsReader to handle multiple layers
> > > - modify graph construction and search algorithm to handle hierarchy
> > >  - run benchmarks
> >
> >
> >
> > --
> > This message was sent by Atlassian Jira
> > (v8.20.10#820010)
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: issues-help@lucene.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Re: [jira] [Commented] (LUCENE-10054) Handle hierarchy in HNSW graph

Posted by David Smiley <ds...@apache.org>.
FYI I had filed https://issues.apache.org/jira/browse/INFRA-23503

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Tue, Jul 26, 2022 at 3:54 PM Michael Sokolov <ms...@gmail.com> wrote:

> searching JIRA for "slkjfdf" I found a few issues in other projects,
> but none seems to be getting the same degree of spam love
>
> On Tue, Jul 26, 2022 at 3:50 PM Mike Sokolov (Jira) <ji...@apache.org>
> wrote:
> >
> >
> >     [
> https://issues.apache.org/jira/browse/LUCENE-10054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571588#comment-17571588
> ]
> >
> > Mike Sokolov commented on LUCENE-10054:
> > ---------------------------------------
> >
> > what is it with this issue that spammers love so much!? I wonder if we
> > could somehow lock it as read-only ...
> >
> >
> >
> > > Handle hierarchy in HNSW graph
> > > ------------------------------
> > >
> > >                 Key: LUCENE-10054
> > >                 URL:
> https://issues.apache.org/jira/browse/LUCENE-10054
> > >             Project: Lucene - Core
> > >          Issue Type: Task
> > >            Reporter: Mayya Sharipova
> > >            Priority: Major
> > >              Labels: vector-based-search
> > >             Fix For: 9.1
> > >
> > >          Time Spent: 20h 20m
> > >  Remaining Estimate: 0h
> > >
> > > Currently HNSW graph is represented as a single layer graph.
> > >  We would like to extend it to handle hierarchy as per [discussion|
> https://issues.apache.org/jira/browse/LUCENE-9004?focusedCommentId=17393216&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17393216
> ].
> > >
> > > TODO tasks:
> > > - add multiple layers in the HnswGraph class
> > >  - modify the format in  Lucene90HnswVectorsWriter and
> Lucene90HnswVectorsReader to handle multiple layers
> > > - modify graph construction and search algorithm to handle hierarchy
> > >  - run benchmarks
> >
> >
> >
> > --
> > This message was sent by Atlassian Jira
> > (v8.20.10#820010)
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: issues-help@lucene.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Re: [jira] [Commented] (LUCENE-10054) Handle hierarchy in HNSW graph

Posted by Michael Sokolov <ms...@gmail.com>.
searching JIRA for "slkjfdf" I found a few issues in other projects,
but none seems to be getting the same degree of spam love

On Tue, Jul 26, 2022 at 3:50 PM Mike Sokolov (Jira) <ji...@apache.org> wrote:
>
>
>     [ https://issues.apache.org/jira/browse/LUCENE-10054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571588#comment-17571588 ]
>
> Mike Sokolov commented on LUCENE-10054:
> ---------------------------------------
>
> what is it with this issue that spammers love so much!? I wonder if we
> could somehow lock it as read-only ...
>
>
>
> > Handle hierarchy in HNSW graph
> > ------------------------------
> >
> >                 Key: LUCENE-10054
> >                 URL: https://issues.apache.org/jira/browse/LUCENE-10054
> >             Project: Lucene - Core
> >          Issue Type: Task
> >            Reporter: Mayya Sharipova
> >            Priority: Major
> >              Labels: vector-based-search
> >             Fix For: 9.1
> >
> >          Time Spent: 20h 20m
> >  Remaining Estimate: 0h
> >
> > Currently HNSW graph is represented as a single layer graph.
> >  We would like to extend it to handle hierarchy as per [discussion|https://issues.apache.org/jira/browse/LUCENE-9004?focusedCommentId=17393216&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17393216].
> >
> > TODO tasks:
> > - add multiple layers in the HnswGraph class
> >  - modify the format in  Lucene90HnswVectorsWriter and Lucene90HnswVectorsReader to handle multiple layers
> > - modify graph construction and search algorithm to handle hierarchy
> >  - run benchmarks
>
>
>
> --
> This message was sent by Atlassian Jira
> (v8.20.10#820010)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
> For additional commands, e-mail: issues-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org