You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "janhoy (via GitHub)" <gi...@apache.org> on 2024/01/03 01:30:06 UTC
[PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
janhoy opened a new pull request, #2175:
URL: https://github.com/apache/solr/pull/2175
https://issues.apache.org/jira/browse/SOLR-16203
Naïve solution, explicitly calling `inform()` in `FieldTypePluginLoader`. The two tests from JIRA are added to this PR to demonstrate.
There may be a more elegant place to do the init. We risk double inform if not using SPI since we don't check that here. Perhaps there is some method somewhere that could be called on the entire chain instead of in each init(), such as the one in `ManagedIndexSchema#informResourceLoaderAwareObjectsInChain()`. I did not dive deep here. Open to suggestions from folks who know the code better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on PR #2175:
URL: https://github.com/apache/solr/pull/2175#issuecomment-1878786136
Thanks for nice proposals @uschindler , the method is now private and code cleaner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on code in PR #2175:
URL: https://github.com/apache/solr/pull/2175#discussion_r1442291279
##########
solr/core/src/java/org/apache/solr/schema/IndexSchema.java:
##########
@@ -1991,6 +1997,37 @@ public IndexSchema deleteCopyFields(Map<String, Collection<String>> copyFields)
throw new SolrException(ErrorCode.SERVER_ERROR, msg);
}
+ /** Informs analyzers used by a fieldType. */
+ protected void informResourceLoaderAwareObjectsForFieldType(FieldType fieldType) {
Review Comment:
This code is *moved* as-is from subclass `ManagedIndexSchema`, which still calls this method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #2175:
URL: https://github.com/apache/solr/pull/2175#discussion_r1442076218
##########
solr/core/src/java/org/apache/solr/schema/IndexSchema.java:
##########
@@ -1991,6 +1997,37 @@ public IndexSchema deleteCopyFields(Map<String, Collection<String>> copyFields)
throw new SolrException(ErrorCode.SERVER_ERROR, msg);
}
+ /** Informs analyzers used by a fieldType. */
+ protected void informResourceLoaderAwareObjectsForFieldType(FieldType fieldType) {
+ // must inform any sub-components used in the
+ // tokenizer chain if they are ResourceLoaderAware
+ if (!fieldType.supportsAnalyzers()) return;
+
+ Analyzer indexAnalyzer = fieldType.getIndexAnalyzer();
Review Comment:
maybe this one works:
```java
var stream = Stream.of(fieldType.getIndexAnalyzer(), fieldType.getQueryAnalyzer());
if (fieldType instanceof TextField) {
stream = Stream.concat(stream, Stream.of(((TextField) fieldType).getMultiTermAnalyzer());
}
stream.filter(TokenizerChain.class::isInstance)
.distinct().map(TokenizerChain.class::cast).foreach(this::informResourceLoaderAwareObjectsInChain);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #2175:
URL: https://github.com/apache/solr/pull/2175#discussion_r1442023598
##########
solr/core/src/java/org/apache/solr/schema/IndexSchema.java:
##########
@@ -1991,6 +1997,37 @@ public IndexSchema deleteCopyFields(Map<String, Collection<String>> copyFields)
throw new SolrException(ErrorCode.SERVER_ERROR, msg);
}
+ /** Informs analyzers used by a fieldType. */
+ protected void informResourceLoaderAwareObjectsForFieldType(FieldType fieldType) {
+ // must inform any sub-components used in the
+ // tokenizer chain if they are ResourceLoaderAware
+ if (!fieldType.supportsAnalyzers()) return;
+
+ Analyzer indexAnalyzer = fieldType.getIndexAnalyzer();
Review Comment:
maybe do something like:
```java
Stream.of(fieldType.getIndexAnalyzer(), fieldType.getQueryAnalyzer(), noIdeaHowToHandleThis.getMultiTermAnalyzer()).filter(a -> a instanceof TokenizerChain)
.distinct().foreach(a -> informResourceLoaderAwareObjectsInChain((TokenizerChain) a);
```
##########
solr/core/src/java/org/apache/solr/schema/IndexSchema.java:
##########
@@ -1991,6 +1997,37 @@ public IndexSchema deleteCopyFields(Map<String, Collection<String>> copyFields)
throw new SolrException(ErrorCode.SERVER_ERROR, msg);
}
+ /** Informs analyzers used by a fieldType. */
+ protected void informResourceLoaderAwareObjectsForFieldType(FieldType fieldType) {
Review Comment:
does this need to be protected? The inner method is private, so why is this one public? A method reference in lambdas does not need to be visible in any way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on PR #2175:
URL: https://github.com/apache/solr/pull/2175#issuecomment-1875084210
Second attempt:
The first attempt did not work for packages, since `FieldTypePluginLoader#init` was too early in the cycle to initialize analyzers from packages.
Managed schema handled the missing `inform` call after SPI already, so moved the method `informResourceLoaderAwareObjectsForFieldType` from `ManagedIndexSchema` to `IndexSchema` and use it at the end of `IndexSchema#readSchema`. All tests pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #2175:
URL: https://github.com/apache/solr/pull/2175#discussion_r1442080975
##########
solr/core/src/java/org/apache/solr/schema/IndexSchema.java:
##########
@@ -1991,6 +1997,37 @@ public IndexSchema deleteCopyFields(Map<String, Collection<String>> copyFields)
throw new SolrException(ErrorCode.SERVER_ERROR, msg);
}
+ /** Informs analyzers used by a fieldType. */
+ protected void informResourceLoaderAwareObjectsForFieldType(FieldType fieldType) {
+ // must inform any sub-components used in the
+ // tokenizer chain if they are ResourceLoaderAware
+ if (!fieldType.supportsAnalyzers()) return;
+
+ Analyzer indexAnalyzer = fieldType.getIndexAnalyzer();
Review Comment:
null check is not needed because of `TokenizerChain.class::isInstance` (returns false on `null`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on code in PR #2175:
URL: https://github.com/apache/solr/pull/2175#discussion_r1442292722
##########
solr/core/src/java/org/apache/solr/schema/IndexSchema.java:
##########
@@ -1991,6 +1997,37 @@ public IndexSchema deleteCopyFields(Map<String, Collection<String>> copyFields)
throw new SolrException(ErrorCode.SERVER_ERROR, msg);
}
+ /** Informs analyzers used by a fieldType. */
+ protected void informResourceLoaderAwareObjectsForFieldType(FieldType fieldType) {
+ // must inform any sub-components used in the
+ // tokenizer chain if they are ResourceLoaderAware
+ if (!fieldType.supportsAnalyzers()) return;
+
+ Analyzer indexAnalyzer = fieldType.getIndexAnalyzer();
Review Comment:
Note that this entire method is *moved* over from `ManagedIndexSchema`, and I have not modified it or attempted to optimize it. May we leave improvements to anoter PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on PR #2175:
URL: https://github.com/apache/solr/pull/2175#issuecomment-1875068088
First attempt in efadf94dc6ac87027ef708c71c58d3848e7a9a0d:
Naïve solution, explicitly calling `inform()` in `FieldTypePluginLoader`. The two tests from JIRA are added to this PR to demonstrate.
There may be a more elegant place to do the init. We risk double inform if not using SPI since we don't check that here. Perhaps there is some method somewhere that could be called on the entire chain instead of in each init(), such as the one in `ManagedIndexSchema#informResourceLoaderAwareObjectsInChain()`. I did not dive deep here. Open to suggestions from folks who know the code better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #2175:
URL: https://github.com/apache/solr/pull/2175#discussion_r1442076218
##########
solr/core/src/java/org/apache/solr/schema/IndexSchema.java:
##########
@@ -1991,6 +1997,37 @@ public IndexSchema deleteCopyFields(Map<String, Collection<String>> copyFields)
throw new SolrException(ErrorCode.SERVER_ERROR, msg);
}
+ /** Informs analyzers used by a fieldType. */
+ protected void informResourceLoaderAwareObjectsForFieldType(FieldType fieldType) {
+ // must inform any sub-components used in the
+ // tokenizer chain if they are ResourceLoaderAware
+ if (!fieldType.supportsAnalyzers()) return;
+
+ Analyzer indexAnalyzer = fieldType.getIndexAnalyzer();
Review Comment:
maybe this one works:
```java
var stream = Stream.of(fieldType.getIndexAnalyzer(), fieldType.getQueryAnalyzer());
if (fieldType instanceof TextField) {
stream = Stream.concat(stream, Stream.of(((TextField) fieldType).getMultiTermAnalyzer());
}
stream.filter(a -> a instanceof TokenizerChain)
.distinct().foreach(a -> informResourceLoaderAwareObjectsInChain((TokenizerChain) a);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #2175:
URL: https://github.com/apache/solr/pull/2175#discussion_r1442078351
##########
solr/core/src/java/org/apache/solr/schema/IndexSchema.java:
##########
@@ -1991,6 +1997,37 @@ public IndexSchema deleteCopyFields(Map<String, Collection<String>> copyFields)
throw new SolrException(ErrorCode.SERVER_ERROR, msg);
}
+ /** Informs analyzers used by a fieldType. */
+ protected void informResourceLoaderAwareObjectsForFieldType(FieldType fieldType) {
+ // must inform any sub-components used in the
+ // tokenizer chain if they are ResourceLoaderAware
+ if (!fieldType.supportsAnalyzers()) return;
+
+ Analyzer indexAnalyzer = fieldType.getIndexAnalyzer();
Review Comment:
(the distinct is by equals, but by default they use system's equals)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on code in PR #2175:
URL: https://github.com/apache/solr/pull/2175#discussion_r1442291279
##########
solr/core/src/java/org/apache/solr/schema/IndexSchema.java:
##########
@@ -1991,6 +1997,37 @@ public IndexSchema deleteCopyFields(Map<String, Collection<String>> copyFields)
throw new SolrException(ErrorCode.SERVER_ERROR, msg);
}
+ /** Informs analyzers used by a fieldType. */
+ protected void informResourceLoaderAwareObjectsForFieldType(FieldType fieldType) {
Review Comment:
This is still also called from `ManagedIndexSchema` class which is a subclass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #2175:
URL: https://github.com/apache/solr/pull/2175#discussion_r1442595590
##########
solr/core/src/java/org/apache/solr/schema/IndexSchema.java:
##########
@@ -652,6 +657,7 @@ protected void readSchema(ConfigSetService.ConfigResource is) {
loadCopyFields(rootNode);
postReadInform();
+ fieldTypes.values().forEach(this::informResourceLoaderAwareObjectsForFieldType);
Review Comment:
move this line into the `postReadInform()` method called before.
##########
solr/core/src/java/org/apache/solr/schema/ManagedIndexSchema.java:
##########
@@ -1307,37 +1299,6 @@ protected void postReadInform() {
}
Review Comment:
remove this whole method, too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #2175:
URL: https://github.com/apache/solr/pull/2175#discussion_r1442590015
##########
solr/core/src/java/org/apache/solr/schema/IndexSchema.java:
##########
@@ -1991,6 +1997,37 @@ public IndexSchema deleteCopyFields(Map<String, Collection<String>> copyFields)
throw new SolrException(ErrorCode.SERVER_ERROR, msg);
}
+ /** Informs analyzers used by a fieldType. */
+ protected void informResourceLoaderAwareObjectsForFieldType(FieldType fieldType) {
+ // must inform any sub-components used in the
+ // tokenizer chain if they are ResourceLoaderAware
+ if (!fieldType.supportsAnalyzers()) return;
+
+ Analyzer indexAnalyzer = fieldType.getIndexAnalyzer();
Review Comment:
Ah, you're right. I just thought al little bit of deduplication of code would be fine.
It's up to you to apply that, but I can make a separate PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on code in PR #2175:
URL: https://github.com/apache/solr/pull/2175#discussion_r1442593581
##########
solr/core/src/java/org/apache/solr/schema/IndexSchema.java:
##########
@@ -1991,6 +1997,37 @@ public IndexSchema deleteCopyFields(Map<String, Collection<String>> copyFields)
throw new SolrException(ErrorCode.SERVER_ERROR, msg);
}
+ /** Informs analyzers used by a fieldType. */
+ protected void informResourceLoaderAwareObjectsForFieldType(FieldType fieldType) {
Review Comment:
This is not ideal. If you move the call to this private method into `postReadInfom()` of *this* class (`IndexSchema`), then the subclass would also call it and it could be private.
Here it is called in readSchema *after* `postReadInform()`. If it is moved there both implementations automatically use it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy merged PR #2175:
URL: https://github.com/apache/solr/pull/2175
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16203: Initialize schema plugins loaded by SPI name [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on code in PR #2175:
URL: https://github.com/apache/solr/pull/2175#discussion_r1442961491
##########
solr/core/src/java/org/apache/solr/schema/IndexSchema.java:
##########
@@ -1991,6 +1997,37 @@ public IndexSchema deleteCopyFields(Map<String, Collection<String>> copyFields)
throw new SolrException(ErrorCode.SERVER_ERROR, msg);
}
+ /** Informs analyzers used by a fieldType. */
+ protected void informResourceLoaderAwareObjectsForFieldType(FieldType fieldType) {
+ // must inform any sub-components used in the
+ // tokenizer chain if they are ResourceLoaderAware
+ if (!fieldType.supportsAnalyzers()) return;
+
+ Analyzer indexAnalyzer = fieldType.getIndexAnalyzer();
Review Comment:
I'll close this comment and defer to new PR after this landed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org