You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Gus Heck (Jira)" <ji...@apache.org> on 2020/09/23 13:34:00 UTC
[jira] [Commented] (SOLR-14787) Inequality support in Payload Check
query parser
[ https://issues.apache.org/jira/browse/SOLR-14787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200828#comment-17200828 ]
Gus Heck commented on SOLR-14787:
---------------------------------
I have found something interesting WRT the failing case you mention... it only fails when I run the test in my IDE. If I use the ant build it passes. I notice some interesting differences in startup for these two scenarios...
build:
{code:java}
[junit4] Suite: org.apache.solr.search.TestPayloadCheckQParserPlugin
[junit4] 2> 1454 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/gus/projects/apache/lucene-solr/fork/lucene-solr8/solr/server/solr/configsets/_default/conf'
[junit4] 2> 1475 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/gus/projects/apache/lucene-solr/fork/lucene-solr8/solr/build/solr-core/test/J0/temp/solr.search.TestPayloadCheckQParserPlugin_AB5E0FC0380BB866-001/data-dir-1-001
[junit4] 2> 1551 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1592 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.e.j.u.log Logging initialized @1620ms to org.eclipse.jetty.util.log.Slf4jLog
[junit4] 2> 1597 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1621 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1626 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 1757 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.7.0
[junit4] 2> 1901 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.s.IndexSchema Schema name=example
[junit4] 2> 1931 WARN (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieIntField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 1936 WARN (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieFloatField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 1940 WARN (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieLongField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 1944 WARN (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieDoubleField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 1966 WARN (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieDateField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 2202 WARN (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.GeoHashField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 2208 WARN (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.LatLonType]. Please consult documentation how to replace it accordingly.
[junit4] 2> 2217 WARN (SUITE-TestPayloadCheckQParserPlugin-seed#[AB5E0FC0380BB866]-worker) [ ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.EnumField]. Please consult documentation how to replace it accordingly.
{code}
IDE (Intellij)
{code:java}
1172 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/gus/projects/apache/lucene-solr/fork/lucene-solr/solr/server/solr/configsets/_default/conf'
1190 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.a.s.SolrTestCaseJ4 Created dataDir: /tmp/solr.search.TestPayloadCheckQParserPlugin_5A2517E33080AEE6-001/data-dir-1-001
1231 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
1252 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.e.j.u.log Logging initialized @1265ms to org.eclipse.jetty.util.log.Slf4jLog
1255 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0)
1269 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=null
1269 WARN (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.a.s.SolrTestCaseJ4 System property java.security.egd was not set by test runner, forcibly set to expected: file:/dev/./urandom
1279 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
1392 WARN (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.a.s.c.SolrConfig You should not use LATEST as luceneMatchVersion property: if you use this setting, and then Solr upgrades to a newer release of Lucene, sizable changes may happen. If precise back compatibility is important then you should instead explicitly specify an actual Lucene version.
1392 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
1518 INFO (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.a.s.s.IndexSchema Schema name=example
1714 WARN (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.GeoHashField]. Please consult documentation how to replace it accordingly.
1719 WARN (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.LatLonType]. Please consult documentation how to replace it accordingly.
1723 WARN (SUITE-TestPayloadCheckQParserPlugin-seed#[5A2517E33080AEE6]-worker) [ ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieDoubleField]. Please consult documentation how to replace it accordingly.
{code}
The ant build is loading a lot more deprecated fields, vs the ide that seems to be picking up Using Lucene MatchVersion: 9.0.0 which is a bit shocking since that shouldn't exist on 8x... [~kwatters] can you check to see if you see something similar in eclipse?
As a side note, the patch when applied to master is passing this test even when the NOUN VERB case is not commented
> Inequality support in Payload Check query parser
> ------------------------------------------------
>
> Key: SOLR-14787
> URL: https://issues.apache.org/jira/browse/SOLR-14787
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Kevin Watters
> Assignee: Gus Heck
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The goal of this ticket/pull request is to support a richer set of matching and filtering based on term payloads. This patch extends the PayloadCheckQueryParser to add a new local param for "op"
> The value of OP could be one of the following
> * gt - greater than
> * gte - greater than or equal
> * lt - less than
> * lte - less than or equal
> default value for "op" if not specified is to be the current behavior of equals.
> Additionally to the operation you can specify a threshold local parameter
> This will provide the ability to search for the term "cat" so long as the payload has a value of greater than 0.75.
> One use case is to classify a document into various categories with an associated confidence or probability that the classification is correct. That can be indexed into a delimited payload field. The searches can find and match documents that were tagged with the "cat" category with a confidence of greater than 0.5.
> Example Document
> {code:java}
> {
> "id":"doc_1",
> "classifications_payload":["cat|0.75 dog|2.0"]
> }
> {code}
> Example Syntax
> {code:java}
> {!payload_check f=classifications_payload payloads='1' op='gt' threshold='0.5'}cat {code}
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org