You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "alessandrobenedetti (via GitHub)" <gi...@apache.org> on 2024/05/21 09:55:57 UTC
[PR] Vectors Format Refactor first draft to improve readability [lucene]
alessandrobenedetti opened a new pull request, #13399:
URL: https://github.com/apache/lucene/pull/13399
### Description
The code for vector formats in the core codec package grew up quite consistently, impacting readability and maintainability.
My main concerns are around duplicate code, package organisation and documentation.
The scope of this pull request is to improve those matters iteratively.
Improvements should be iterative so I'll keep this pull request as small as possible and then we can iteratively build on it.
N.B. This contribution is in draft but I want to collect some first feedback to adapt it
N.B. This contribution is a side effect of a workshop I'm preparing for Community Over Code EU 2024, so it's still missing formalities such as package documentation, Javadocs, polishment but I strongly encourage to participate so that we can finalise it soon (this area is moving fast)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Vectors Format Refactor to improve readability [lucene]
Posted by "alessandrobenedetti (via GitHub)" <gi...@apache.org>.
alessandrobenedetti commented on PR #13399:
URL: https://github.com/apache/lucene/pull/13399#issuecomment-2122366365
Obviously no hard opinion in naming sub-packages or how to group classes, but my feeling is that the general audience would benefit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Vectors Format Refactor to improve readability [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #13399:
URL: https://github.com/apache/lucene/pull/13399#issuecomment-2122338445
Does it actually improve readability? I know some Java projects like to be very granular in how they organize packages, but I've come to like Lucene's relatively flat package structure, like Java's. This way, we can more often take advantage of pkg-private visibility, and packages tend to be more meaningful and have proper javadocs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Vectors Format Refactor to improve readability [lucene]
Posted by "alessandrobenedetti (via GitHub)" <gi...@apache.org>.
alessandrobenedetti commented on PR #13399:
URL: https://github.com/apache/lucene/pull/13399#issuecomment-2122364895
> Does it actually improve readability? I know some Java projects like to be very granular in how they organize packages, but I've come to like Lucene's relatively flat package structure, like Java's. This way, we can more often take advantage of pkg-private visibility, and packages tend to be more meaningful and have proper javadocs.
While preparing the workshop, it took some time to understand that we have now formats dedicated to storing the graph and that internally use other formats to store the actual vectors. With such a flat structure it was hard to understand where there was a choice (like a quantized or not quantized format) or where the formats were solving different problems (storing the graph vs storing the actual vectors).
And I am also quite familiar with the vector-based search in Lucene.
I can imagine a newcomer, even decently expert in Lucene or Java to struggle to reconcile a huge package with a lot of classes, where it's not clear who does what.
Also some of the formats, have a minimal difference (like the HNSW quantised vs not quantised), but the decently big overlapping code make it harder to read (there for example the only difference is basically one instance variable)
Anyway, as I progress in preparing the workshop I may add here additional considerations and tweaks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org