You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "donnerpeter (via GitHub)" <gi...@apache.org> on 2023/07/26 19:42:13 UTC
[GitHub] [lucene] donnerpeter opened a new pull request, #12464: hunspell: make the hash table load factor customizable
donnerpeter opened a new pull request, #12464:
URL: https://github.com/apache/lucene/pull/12464
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] donnerpeter commented on a diff in pull request #12464: hunspell: make the hash table load factor customizable
Posted by "donnerpeter (via GitHub)" <gi...@apache.org>.
donnerpeter commented on code in PR #12464:
URL: https://github.com/apache/lucene/pull/12464#discussion_r1276257435
##########
lucene/analysis/common/src/java/org/apache/lucene/analysis/hunspell/Dictionary.java:
##########
@@ -1204,6 +1205,16 @@ char caseFold(char c) {
}
}
+ /**
+ * The factor determining the size of the internal hash table used for storing the entries. The
+ * table size is {@code entry_count * hashFactor}. The default factor is 1.0. If there are too
+ * many hash collisions, the factor can be increased, resulting in faster access, but more memory
+ * usage.
+ */
+ protected static double hashFactor() {
+ return 1.0;
Review Comment:
Ouch. Late night commits have some disadvantages :) Thanks for spotting!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] dweiss commented on a diff in pull request #12464: hunspell: make the hash table load factor customizable
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on code in PR #12464:
URL: https://github.com/apache/lucene/pull/12464#discussion_r1277381583
##########
lucene/analysis/common/src/java/org/apache/lucene/analysis/hunspell/WordStorage.java:
##########
@@ -390,7 +391,7 @@ private int flushGroup() throws IOException {
if (++chainLengths[hash] > 20) {
throw new RuntimeException(
- "Too many collisions, please report this to dev@lucene.apache.org");
Review Comment:
Yep, something like it. Maybe even provide some diagnostics?:
"Too many collisions. Try a larger Dictionary#hashFactor (now: " + hashFactor() + "). If this doesn't help, please report this to dev@lucene.apache.org"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] donnerpeter merged pull request #12464: hunspell: make the hash table load factor customizable
Posted by "donnerpeter (via GitHub)" <gi...@apache.org>.
donnerpeter merged PR #12464:
URL: https://github.com/apache/lucene/pull/12464
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] dweiss commented on a diff in pull request #12464: hunspell: make the hash table load factor customizable
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on code in PR #12464:
URL: https://github.com/apache/lucene/pull/12464#discussion_r1276255030
##########
lucene/analysis/common/src/java/org/apache/lucene/analysis/hunspell/Dictionary.java:
##########
@@ -1204,6 +1205,16 @@ char caseFold(char c) {
}
}
+ /**
+ * The factor determining the size of the internal hash table used for storing the entries. The
+ * table size is {@code entry_count * hashFactor}. The default factor is 1.0. If there are too
+ * many hash collisions, the factor can be increased, resulting in faster access, but more memory
+ * usage.
+ */
+ protected static double hashFactor() {
+ return 1.0;
Review Comment:
Sorry for the dumb question but how can this be increased without bytecode manipulation or source code changes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] donnerpeter commented on a diff in pull request #12464: hunspell: make the hash table load factor customizable
Posted by "donnerpeter (via GitHub)" <gi...@apache.org>.
donnerpeter commented on code in PR #12464:
URL: https://github.com/apache/lucene/pull/12464#discussion_r1276257435
##########
lucene/analysis/common/src/java/org/apache/lucene/analysis/hunspell/Dictionary.java:
##########
@@ -1204,6 +1205,16 @@ char caseFold(char c) {
}
}
+ /**
+ * The factor determining the size of the internal hash table used for storing the entries. The
+ * table size is {@code entry_count * hashFactor}. The default factor is 1.0. If there are too
+ * many hash collisions, the factor can be increased, resulting in faster access, but more memory
+ * usage.
+ */
+ protected static double hashFactor() {
+ return 1.0;
Review Comment:
Ouch. Late night commits have some disadvantages :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] donnerpeter commented on a diff in pull request #12464: hunspell: make the hash table load factor customizable
Posted by "donnerpeter (via GitHub)" <gi...@apache.org>.
donnerpeter commented on code in PR #12464:
URL: https://github.com/apache/lucene/pull/12464#discussion_r1276455206
##########
lucene/analysis/common/src/java/org/apache/lucene/analysis/hunspell/WordStorage.java:
##########
@@ -390,7 +391,7 @@ private int flushGroup() throws IOException {
if (++chainLengths[hash] > 20) {
throw new RuntimeException(
- "Too many collisions, please report this to dev@lucene.apache.org");
Review Comment:
Something like this? `Too many collisions. Try a larger Dictionary#hashFactor. If this doesn't help, please report this to dev@lucene.apache.org`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] dweiss commented on a diff in pull request #12464: hunspell: make the hash table load factor customizable
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on code in PR #12464:
URL: https://github.com/apache/lucene/pull/12464#discussion_r1276354832
##########
lucene/analysis/common/src/java/org/apache/lucene/analysis/hunspell/WordStorage.java:
##########
@@ -390,7 +391,7 @@ private int flushGroup() throws IOException {
if (++chainLengths[hash] > 20) {
throw new RuntimeException(
- "Too many collisions, please report this to dev@lucene.apache.org");
Review Comment:
I wonder whether this message should remain as it was. Most people will use it out of the box and we can then count on their feedback if something is not right. Those who do override the default will certainly know where to look (we can report the hashFactor as part of the exception message, perhaps)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org