You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Jörn Franke <jo...@gmail.com> on 2019/11/06 21:14:00 UTC
Re: Solr 8.3 Solrj streaming expressions do not return all field values
I created a JIRA for this:
https://issues.apache.org/jira/browse/SOLR-13894
On Wed, Nov 6, 2019 at 10:45 AM Jörn Franke <jo...@gmail.com> wrote:
> I have checked now Solr 8.3 server in admin UI. Same issue.
>
> Reproduction:
> select(search(testcollection,q=“test”,df=“Default”,defType=“edismax”,fl=“id”,
> qt=“/export”, sort=“id asc”),id,if(eq(1,1),Y,N) as found)
>
> In 8.3 it returns only the id field.
> In 8.2 it returns id,found field.
>
> Since found is generated by select (and not coming from the collection)
> there must be an issue with select.
>
> Any idea why this is happening.
>
> Debug logs do not show any error and the expression is correctly received
> by Solr.
>
> Thank you.
>
> Best regards
>
> > Am 05.11.2019 um 14:59 schrieb Jörn Franke <jo...@gmail.com>:
> >
> > Thanks I will check and come back to you. As far as I remember (but
> have to check) the queries generated by Solr were correct
> >
> > Just to be clear the same thing works with Solr 8.2 server and Solr 8.2
> client.
> >
> > It show the odd behaviour with Solr 8.2 server and Solr 8.3 client.
> >
> >> Am 05.11.2019 um 14:49 schrieb Joel Bernstein <jo...@gmail.com>:
> >>
> >> I'll probably need some more details. One thing that's useful is to
> look at
> >> the logs and see the underlying Solr queries that are generated. Then
> try
> >> those underlying queries against the Solr index and see what comes
> back. If
> >> you're not seeing the fields with the plain Solr queries then we know
> it's
> >> something going on below streaming expressions. If you are seeing the
> >> fields then it's the expressions themselves that are not handling the
> data
> >> as expected.
> >>
> >>
> >> Joel Bernstein
> >> http://joelsolr.blogspot.com/
> >>
> >>
> >>>> On Mon, Nov 4, 2019 at 9:09 AM Jörn Franke <jo...@gmail.com>
> wrote:
> >>>
> >>> Most likely this issue can bei also reproduced in the admin UI for the
> >>> streaming handler of a collection.
> >>>
> >>>>> Am 04.11.2019 um 13:32 schrieb Jörn Franke <jo...@gmail.com>:
> >>>>
> >>>> Hi,
> >>>>
> >>>> I use streaming expressions, e.g.
> >>>> Sort(Select(search(...),id,if(eq(1,1),Y,N) as found), by=“field A
> asc”)
> >>>> (Using export handler, sort is not really mandatory , I will remove it
> >>> later anyway)
> >>>>
> >>>> This works perfectly fine if I use Solr 8.2.0 (server + client). It
> >>> returns Tuples in the form { “id”,”12345”, “found”:”Y”}
> >>>>
> >>>> However, if I use Solr 8.2.0 as server and Solr 8.3.0 as client then
> the
> >>> above statement only returns the id field, but not the found field.
> >>>>
> >>>> Questions:
> >>>> 1) is this expected behavior, ie Solr client 8.3.0 is in this case not
> >>> compatible with Solr 8.2.0 and server upgrade to Solr 8.3.0 will fix
> this?
> >>>> 2) has the syntax for the above expression changed? If so how?
> >>>> 3) is this not expected behavior and I should create a Jira for it?
> >>>>
> >>>> Thank you.
> >>>> Best regards
> >>>
>
Re: Solr 8.3 Solrj streaming expressions do not return all field values
Posted by Jörn Franke <jo...@gmail.com>.
No problem, but thanks a lot for the feedback and considerations. I will try to get into the process on how to update the Solr ref guide. I found a lot of things that I could solve to facilitate others to do a solution based on Solr.
> Am 20.11.2019 um 13:04 schrieb Joel Bernstein <jo...@gmail.com>:
>
>
> Ah, I should have noticed that. Yes, there was a change in how string literals are handled. Quotes are now required for string literals in Math Expressions. I suspect this change is going to cause problems for other people as well, but it needed to be done to clarify aspects of the language. Sorry for the confusion. I'll make a point of updating the documentation to make sure all examples in the documentation are correct.
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
>> On Tue, Nov 19, 2019 at 5:56 PM Jörn Franke <jo...@gmail.com> wrote:
>> Ok, solved. Solr 8.2 accepted this statement:
>> select(search(testcollection,q="test",df="Default",defType="edismax",fl="id", qt="/export", sort="id asc"),id,if(eq(1,1),Y,N) as found)
>>
>> and return to me the expected results. Note that around Y and N there is no ". Solr 8.3 requires the following statement
>> select(search(testcollection,q="test",df="Default",defType="edismax",fl="id", qt="/export", sort="id asc"),id,if(eq(1,1),"Y","N") as found)
>> to return the exact results. I do not know why I did originally not include the quotation marks around Y and N, but it seems that 8.2 accepted this and 8.3 not. I will update the Jira.
>>
>>
>>
>>> On Tue, Nov 19, 2019 at 11:25 PM Jörn Franke <jo...@gmail.com> wrote:
>>> Could it be that Solr 8.3 is more strict on the if statement?
>>> the statement if(eq(1,1),Y,N)
>>> is supposed to return the character Y (not the field). In Solr 8.2 it returns the character Y, but in Solr 8.3 not.
>>>
>>>> On Tue, Nov 19, 2019 at 11:21 PM Jörn Franke <jo...@gmail.com> wrote:
>>>> thanks i will find another installation of Solr, but last time the underlying queries to the export handler were correct. What was wrong was the generated field "found" based on the if statement based NOT on any data in the collectioj. As said, in Solr 8.3 the id fields are returned, but not the "found" field, which is generated by the selected statement:
>>>> select(search(testcollection,q="test",df="Default",defType="edismax",fl="id", qt="/export", sort="id asc"),id,if(eq(1,1),Y,N) as found)
>>>>
>>>> returns a tuple
>>>> id: "blabla"
>>>>
>>>> but not the part generated by the if(eq(1,1),Y,N) as found. In Solr 8.2 the found field is returned:
>>>> id: "blabla"
>>>> found: "Y"
>>>> To my best knowledge this is a fully generated field => it does not depend on the underlying collection data and the export handler. This was done to exclude any issues with this one.
>>>> it states simply if 1=1 return Y else N
>>>>
>>>>
>>>>> On Fri, Nov 8, 2019 at 3:20 PM Joel Bernstein <jo...@gmail.com> wrote:
>>>>> Before moving to a jira let's take a look at the underlying Solr queries in the log. The Streaming Expressions just creates solr queries, in this case queries to the /export handler. So when something is not working as expected we want to strip away the streaming expression and debug the actual queries that are being run.
>>>>>
>>>>> You can find the Solr queries that appear in the log after running the expressions and then try running them outside of the expression as plain Solr queries.
>>>>>
>>>>> You can also post the Solr queries to this thread and we discuss what the logs say.
>>>>>
>>>>> In these cases the logs always are the way to debug whats going on.
>>>>>
>>>>>
>>>>> Joel Bernstein
>>>>> http://joelsolr.blogspot.com/
>>>>>
>>>>>
>>>>>> On Wed, Nov 6, 2019 at 4:14 PM Jörn Franke <jo...@gmail.com> wrote:
>>>>>> I created a JIRA for this:
>>>>>> https://issues.apache.org/jira/browse/SOLR-13894
>>>>>>
>>>>>>> On Wed, Nov 6, 2019 at 10:45 AM Jörn Franke <jo...@gmail.com> wrote:
>>>>>>> I have checked now Solr 8.3 server in admin UI. Same issue.
>>>>>>>
>>>>>>> Reproduction:
>>>>>>> select(search(testcollection,q=“test”,df=“Default”,defType=“edismax”,fl=“id”, qt=“/export”, sort=“id asc”),id,if(eq(1,1),Y,N) as found)
>>>>>>>
>>>>>>> In 8.3 it returns only the id field.
>>>>>>> In 8.2 it returns id,found field.
>>>>>>>
>>>>>>> Since found is generated by select (and not coming from the collection) there must be an issue with select.
>>>>>>>
>>>>>>> Any idea why this is happening.
>>>>>>>
>>>>>>> Debug logs do not show any error and the expression is correctly received by Solr.
>>>>>>>
>>>>>>> Thank you.
>>>>>>>
>>>>>>> Best regards
>>>>>>>
>>>>>>> > Am 05.11.2019 um 14:59 schrieb Jörn Franke <jo...@gmail.com>:
>>>>>>> >
>>>>>>> > Thanks I will check and come back to you. As far as I remember (but have to check) the queries generated by Solr were correct
>>>>>>> >
>>>>>>> > Just to be clear the same thing works with Solr 8.2 server and Solr 8.2 client.
>>>>>>> >
>>>>>>> > It show the odd behaviour with Solr 8.2 server and Solr 8.3 client.
>>>>>>> >
>>>>>>> >> Am 05.11.2019 um 14:49 schrieb Joel Bernstein <jo...@gmail.com>:
>>>>>>> >>
>>>>>>> >> I'll probably need some more details. One thing that's useful is to look at
>>>>>>> >> the logs and see the underlying Solr queries that are generated. Then try
>>>>>>> >> those underlying queries against the Solr index and see what comes back. If
>>>>>>> >> you're not seeing the fields with the plain Solr queries then we know it's
>>>>>>> >> something going on below streaming expressions. If you are seeing the
>>>>>>> >> fields then it's the expressions themselves that are not handling the data
>>>>>>> >> as expected.
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> Joel Bernstein
>>>>>>> >> http://joelsolr.blogspot.com/
>>>>>>> >>
>>>>>>> >>
>>>>>>> >>>> On Mon, Nov 4, 2019 at 9:09 AM Jörn Franke <jo...@gmail.com> wrote:
>>>>>>> >>>
>>>>>>> >>> Most likely this issue can bei also reproduced in the admin UI for the
>>>>>>> >>> streaming handler of a collection.
>>>>>>> >>>
>>>>>>> >>>>> Am 04.11.2019 um 13:32 schrieb Jörn Franke <jo...@gmail.com>:
>>>>>>> >>>>
>>>>>>> >>>> Hi,
>>>>>>> >>>>
>>>>>>> >>>> I use streaming expressions, e.g.
>>>>>>> >>>> Sort(Select(search(...),id,if(eq(1,1),Y,N) as found), by=“field A asc”)
>>>>>>> >>>> (Using export handler, sort is not really mandatory , I will remove it
>>>>>>> >>> later anyway)
>>>>>>> >>>>
>>>>>>> >>>> This works perfectly fine if I use Solr 8.2.0 (server + client). It
>>>>>>> >>> returns Tuples in the form { “id”,”12345”, “found”:”Y”}
>>>>>>> >>>>
>>>>>>> >>>> However, if I use Solr 8.2.0 as server and Solr 8.3.0 as client then the
>>>>>>> >>> above statement only returns the id field, but not the found field.
>>>>>>> >>>>
>>>>>>> >>>> Questions:
>>>>>>> >>>> 1) is this expected behavior, ie Solr client 8.3.0 is in this case not
>>>>>>> >>> compatible with Solr 8.2.0 and server upgrade to Solr 8.3.0 will fix this?
>>>>>>> >>>> 2) has the syntax for the above expression changed? If so how?
>>>>>>> >>>> 3) is this not expected behavior and I should create a Jira for it?
>>>>>>> >>>>
>>>>>>> >>>> Thank you.
>>>>>>> >>>> Best regards
>>>>>>> >>>
Re: Solr 8.3 Solrj streaming expressions do not return all field values
Posted by Joel Bernstein <jo...@gmail.com>.
Ah, I should have noticed that. Yes, there was a change in how string
literals are handled. Quotes are now required for string literals in Math
Expressions. I suspect this change is going to cause problems for other
people as well, but it needed to be done to clarify aspects of the
language. Sorry for the confusion. I'll make a point of updating the
documentation to make sure all examples in the documentation are correct.
Joel Bernstein
http://joelsolr.blogspot.com/
On Tue, Nov 19, 2019 at 5:56 PM Jörn Franke <jo...@gmail.com> wrote:
> Ok, solved. Solr 8.2 accepted this statement:
> select(search(testcollection,q="test",df="Default",defType="edismax",fl=
> "id", qt="/export", sort="id asc"),id,if(eq(1,1),Y,N) as found)
>
> and return to me the expected results. Note that around Y and N there is
> no ". Solr 8.3 requires the following statement
> select(search(testcollection,q="test",df="Default",defType="edismax",fl=
> "id", qt="/export", sort="id asc"),id,if(eq(1,1),"Y","N") as found)
> to return the exact results. I do not know why I did originally not
> include the quotation marks around Y and N, but it seems that 8.2 accepted
> this and 8.3 not. I will update the Jira.
>
>
>
> On Tue, Nov 19, 2019 at 11:25 PM Jörn Franke <jo...@gmail.com> wrote:
>
>> Could it be that Solr 8.3 is more strict on the if statement?
>> the statement if(eq(1,1),Y,N)
>> is supposed to return the character Y (not the field). In Solr 8.2 it
>> returns the character Y, but in Solr 8.3 not.
>>
>> On Tue, Nov 19, 2019 at 11:21 PM Jörn Franke <jo...@gmail.com>
>> wrote:
>>
>>> thanks i will find another installation of Solr, but last time the
>>> underlying queries to the export handler were correct. What was wrong was
>>> the generated field "found" based on the if statement based NOT on any data
>>> in the collectioj. As said, in Solr 8.3 the id fields are returned, but
>>> not the "found" field, which is generated by the selected statement:
>>> select(search(testcollection,q="test",df="Default",defType="edismax",fl=
>>> "id", qt="/export", sort="id asc"),id,if(eq(1,1),Y,N) as found)
>>>
>>> returns a tuple
>>> id: "blabla"
>>>
>>> but not the part generated by the if(eq(1,1),Y,N) as found. In Solr 8.2
>>> the found field is returned:
>>> id: "blabla"
>>> found: "Y"
>>> To my best knowledge this is a fully generated field => it does not
>>> depend on the underlying collection data and the export handler. This was
>>> done to exclude any issues with this one.
>>> it states simply if 1=1 return Y else N
>>>
>>>
>>> On Fri, Nov 8, 2019 at 3:20 PM Joel Bernstein <jo...@gmail.com>
>>> wrote:
>>>
>>>> Before moving to a jira let's take a look at the underlying Solr
>>>> queries in the log. The Streaming Expressions just creates solr queries, in
>>>> this case queries to the /export handler. So when something is not working
>>>> as expected we want to strip away the streaming expression and debug the
>>>> actual queries that are being run.
>>>>
>>>> You can find the Solr queries that appear in the log after running the
>>>> expressions and then try running them outside of the expression as plain
>>>> Solr queries.
>>>>
>>>> You can also post the Solr queries to this thread and we discuss what
>>>> the logs say.
>>>>
>>>> In these cases the logs always are the way to debug whats going on.
>>>>
>>>>
>>>> Joel Bernstein
>>>> http://joelsolr.blogspot.com/
>>>>
>>>>
>>>> On Wed, Nov 6, 2019 at 4:14 PM Jörn Franke <jo...@gmail.com>
>>>> wrote:
>>>>
>>>>> I created a JIRA for this:
>>>>> https://issues.apache.org/jira/browse/SOLR-13894
>>>>>
>>>>> On Wed, Nov 6, 2019 at 10:45 AM Jörn Franke <jo...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I have checked now Solr 8.3 server in admin UI. Same issue.
>>>>>>
>>>>>> Reproduction:
>>>>>> select(search(testcollection,q=“test”,df=“Default”,defType=“edismax”,fl=“id”,
>>>>>> qt=“/export”, sort=“id asc”),id,if(eq(1,1),Y,N) as found)
>>>>>>
>>>>>> In 8.3 it returns only the id field.
>>>>>> In 8.2 it returns id,found field.
>>>>>>
>>>>>> Since found is generated by select (and not coming from the
>>>>>> collection) there must be an issue with select.
>>>>>>
>>>>>> Any idea why this is happening.
>>>>>>
>>>>>> Debug logs do not show any error and the expression is correctly
>>>>>> received by Solr.
>>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>> Best regards
>>>>>>
>>>>>> > Am 05.11.2019 um 14:59 schrieb Jörn Franke <jo...@gmail.com>:
>>>>>> >
>>>>>> > Thanks I will check and come back to you. As far as I remember
>>>>>> (but have to check) the queries generated by Solr were correct
>>>>>> >
>>>>>> > Just to be clear the same thing works with Solr 8.2 server and Solr
>>>>>> 8.2 client.
>>>>>> >
>>>>>> > It show the odd behaviour with Solr 8.2 server and Solr 8.3 client.
>>>>>> >
>>>>>> >> Am 05.11.2019 um 14:49 schrieb Joel Bernstein <joelsolr@gmail.com
>>>>>> >:
>>>>>> >>
>>>>>> >> I'll probably need some more details. One thing that's useful is
>>>>>> to look at
>>>>>> >> the logs and see the underlying Solr queries that are generated.
>>>>>> Then try
>>>>>> >> those underlying queries against the Solr index and see what comes
>>>>>> back. If
>>>>>> >> you're not seeing the fields with the plain Solr queries then we
>>>>>> know it's
>>>>>> >> something going on below streaming expressions. If you are seeing
>>>>>> the
>>>>>> >> fields then it's the expressions themselves that are not handling
>>>>>> the data
>>>>>> >> as expected.
>>>>>> >>
>>>>>> >>
>>>>>> >> Joel Bernstein
>>>>>> >> http://joelsolr.blogspot.com/
>>>>>> >>
>>>>>> >>
>>>>>> >>>> On Mon, Nov 4, 2019 at 9:09 AM Jörn Franke <jo...@gmail.com>
>>>>>> wrote:
>>>>>> >>>
>>>>>> >>> Most likely this issue can bei also reproduced in the admin UI
>>>>>> for the
>>>>>> >>> streaming handler of a collection.
>>>>>> >>>
>>>>>> >>>>> Am 04.11.2019 um 13:32 schrieb Jörn Franke <
>>>>>> jornfranke@gmail.com>:
>>>>>> >>>>
>>>>>> >>>> Hi,
>>>>>> >>>>
>>>>>> >>>> I use streaming expressions, e.g.
>>>>>> >>>> Sort(Select(search(...),id,if(eq(1,1),Y,N) as found), by=“field
>>>>>> A asc”)
>>>>>> >>>> (Using export handler, sort is not really mandatory , I will
>>>>>> remove it
>>>>>> >>> later anyway)
>>>>>> >>>>
>>>>>> >>>> This works perfectly fine if I use Solr 8.2.0 (server + client).
>>>>>> It
>>>>>> >>> returns Tuples in the form { “id”,”12345”, “found”:”Y”}
>>>>>> >>>>
>>>>>> >>>> However, if I use Solr 8.2.0 as server and Solr 8.3.0 as client
>>>>>> then the
>>>>>> >>> above statement only returns the id field, but not the found
>>>>>> field.
>>>>>> >>>>
>>>>>> >>>> Questions:
>>>>>> >>>> 1) is this expected behavior, ie Solr client 8.3.0 is in this
>>>>>> case not
>>>>>> >>> compatible with Solr 8.2.0 and server upgrade to Solr 8.3.0 will
>>>>>> fix this?
>>>>>> >>>> 2) has the syntax for the above expression changed? If so how?
>>>>>> >>>> 3) is this not expected behavior and I should create a Jira for
>>>>>> it?
>>>>>> >>>>
>>>>>> >>>> Thank you.
>>>>>> >>>> Best regards
>>>>>> >>>
>>>>>>
>>>>>
Re: Solr 8.3 Solrj streaming expressions do not return all field values
Posted by Jörn Franke <jo...@gmail.com>.
Ok, solved. Solr 8.2 accepted this statement:
select(search(testcollection,q="test",df="Default",defType="edismax",fl="id",
qt="/export", sort="id asc"),id,if(eq(1,1),Y,N) as found)
and return to me the expected results. Note that around Y and N there is no
". Solr 8.3 requires the following statement
select(search(testcollection,q="test",df="Default",defType="edismax",fl="id",
qt="/export", sort="id asc"),id,if(eq(1,1),"Y","N") as found)
to return the exact results. I do not know why I did originally not include
the quotation marks around Y and N, but it seems that 8.2 accepted this and
8.3 not. I will update the Jira.
On Tue, Nov 19, 2019 at 11:25 PM Jörn Franke <jo...@gmail.com> wrote:
> Could it be that Solr 8.3 is more strict on the if statement?
> the statement if(eq(1,1),Y,N)
> is supposed to return the character Y (not the field). In Solr 8.2 it
> returns the character Y, but in Solr 8.3 not.
>
> On Tue, Nov 19, 2019 at 11:21 PM Jörn Franke <jo...@gmail.com> wrote:
>
>> thanks i will find another installation of Solr, but last time the
>> underlying queries to the export handler were correct. What was wrong was
>> the generated field "found" based on the if statement based NOT on any data
>> in the collectioj. As said, in Solr 8.3 the id fields are returned, but
>> not the "found" field, which is generated by the selected statement:
>> select(search(testcollection,q="test",df="Default",defType="edismax",fl=
>> "id", qt="/export", sort="id asc"),id,if(eq(1,1),Y,N) as found)
>>
>> returns a tuple
>> id: "blabla"
>>
>> but not the part generated by the if(eq(1,1),Y,N) as found. In Solr 8.2
>> the found field is returned:
>> id: "blabla"
>> found: "Y"
>> To my best knowledge this is a fully generated field => it does not
>> depend on the underlying collection data and the export handler. This was
>> done to exclude any issues with this one.
>> it states simply if 1=1 return Y else N
>>
>>
>> On Fri, Nov 8, 2019 at 3:20 PM Joel Bernstein <jo...@gmail.com> wrote:
>>
>>> Before moving to a jira let's take a look at the underlying Solr queries
>>> in the log. The Streaming Expressions just creates solr queries, in this
>>> case queries to the /export handler. So when something is not working as
>>> expected we want to strip away the streaming expression and debug the
>>> actual queries that are being run.
>>>
>>> You can find the Solr queries that appear in the log after running the
>>> expressions and then try running them outside of the expression as plain
>>> Solr queries.
>>>
>>> You can also post the Solr queries to this thread and we discuss what
>>> the logs say.
>>>
>>> In these cases the logs always are the way to debug whats going on.
>>>
>>>
>>> Joel Bernstein
>>> http://joelsolr.blogspot.com/
>>>
>>>
>>> On Wed, Nov 6, 2019 at 4:14 PM Jörn Franke <jo...@gmail.com> wrote:
>>>
>>>> I created a JIRA for this:
>>>> https://issues.apache.org/jira/browse/SOLR-13894
>>>>
>>>> On Wed, Nov 6, 2019 at 10:45 AM Jörn Franke <jo...@gmail.com>
>>>> wrote:
>>>>
>>>>> I have checked now Solr 8.3 server in admin UI. Same issue.
>>>>>
>>>>> Reproduction:
>>>>> select(search(testcollection,q=“test”,df=“Default”,defType=“edismax”,fl=“id”,
>>>>> qt=“/export”, sort=“id asc”),id,if(eq(1,1),Y,N) as found)
>>>>>
>>>>> In 8.3 it returns only the id field.
>>>>> In 8.2 it returns id,found field.
>>>>>
>>>>> Since found is generated by select (and not coming from the
>>>>> collection) there must be an issue with select.
>>>>>
>>>>> Any idea why this is happening.
>>>>>
>>>>> Debug logs do not show any error and the expression is correctly
>>>>> received by Solr.
>>>>>
>>>>> Thank you.
>>>>>
>>>>> Best regards
>>>>>
>>>>> > Am 05.11.2019 um 14:59 schrieb Jörn Franke <jo...@gmail.com>:
>>>>> >
>>>>> > Thanks I will check and come back to you. As far as I remember (but
>>>>> have to check) the queries generated by Solr were correct
>>>>> >
>>>>> > Just to be clear the same thing works with Solr 8.2 server and Solr
>>>>> 8.2 client.
>>>>> >
>>>>> > It show the odd behaviour with Solr 8.2 server and Solr 8.3 client.
>>>>> >
>>>>> >> Am 05.11.2019 um 14:49 schrieb Joel Bernstein <jo...@gmail.com>:
>>>>> >>
>>>>> >> I'll probably need some more details. One thing that's useful is
>>>>> to look at
>>>>> >> the logs and see the underlying Solr queries that are generated.
>>>>> Then try
>>>>> >> those underlying queries against the Solr index and see what comes
>>>>> back. If
>>>>> >> you're not seeing the fields with the plain Solr queries then we
>>>>> know it's
>>>>> >> something going on below streaming expressions. If you are seeing
>>>>> the
>>>>> >> fields then it's the expressions themselves that are not handling
>>>>> the data
>>>>> >> as expected.
>>>>> >>
>>>>> >>
>>>>> >> Joel Bernstein
>>>>> >> http://joelsolr.blogspot.com/
>>>>> >>
>>>>> >>
>>>>> >>>> On Mon, Nov 4, 2019 at 9:09 AM Jörn Franke <jo...@gmail.com>
>>>>> wrote:
>>>>> >>>
>>>>> >>> Most likely this issue can bei also reproduced in the admin UI for
>>>>> the
>>>>> >>> streaming handler of a collection.
>>>>> >>>
>>>>> >>>>> Am 04.11.2019 um 13:32 schrieb Jörn Franke <jornfranke@gmail.com
>>>>> >:
>>>>> >>>>
>>>>> >>>> Hi,
>>>>> >>>>
>>>>> >>>> I use streaming expressions, e.g.
>>>>> >>>> Sort(Select(search(...),id,if(eq(1,1),Y,N) as found), by=“field A
>>>>> asc”)
>>>>> >>>> (Using export handler, sort is not really mandatory , I will
>>>>> remove it
>>>>> >>> later anyway)
>>>>> >>>>
>>>>> >>>> This works perfectly fine if I use Solr 8.2.0 (server + client).
>>>>> It
>>>>> >>> returns Tuples in the form { “id”,”12345”, “found”:”Y”}
>>>>> >>>>
>>>>> >>>> However, if I use Solr 8.2.0 as server and Solr 8.3.0 as client
>>>>> then the
>>>>> >>> above statement only returns the id field, but not the found field.
>>>>> >>>>
>>>>> >>>> Questions:
>>>>> >>>> 1) is this expected behavior, ie Solr client 8.3.0 is in this
>>>>> case not
>>>>> >>> compatible with Solr 8.2.0 and server upgrade to Solr 8.3.0 will
>>>>> fix this?
>>>>> >>>> 2) has the syntax for the above expression changed? If so how?
>>>>> >>>> 3) is this not expected behavior and I should create a Jira for
>>>>> it?
>>>>> >>>>
>>>>> >>>> Thank you.
>>>>> >>>> Best regards
>>>>> >>>
>>>>>
>>>>
Re: Solr 8.3 Solrj streaming expressions do not return all field values
Posted by Jörn Franke <jo...@gmail.com>.
Could it be that Solr 8.3 is more strict on the if statement?
the statement if(eq(1,1),Y,N)
is supposed to return the character Y (not the field). In Solr 8.2 it
returns the character Y, but in Solr 8.3 not.
On Tue, Nov 19, 2019 at 11:21 PM Jörn Franke <jo...@gmail.com> wrote:
> thanks i will find another installation of Solr, but last time the
> underlying queries to the export handler were correct. What was wrong was
> the generated field "found" based on the if statement based NOT on any data
> in the collectioj. As said, in Solr 8.3 the id fields are returned, but
> not the "found" field, which is generated by the selected statement:
> select(search(testcollection,q="test",df="Default",defType="edismax",fl=
> "id", qt="/export", sort="id asc"),id,if(eq(1,1),Y,N) as found)
>
> returns a tuple
> id: "blabla"
>
> but not the part generated by the if(eq(1,1),Y,N) as found. In Solr 8.2
> the found field is returned:
> id: "blabla"
> found: "Y"
> To my best knowledge this is a fully generated field => it does not depend
> on the underlying collection data and the export handler. This was done to
> exclude any issues with this one.
> it states simply if 1=1 return Y else N
>
>
> On Fri, Nov 8, 2019 at 3:20 PM Joel Bernstein <jo...@gmail.com> wrote:
>
>> Before moving to a jira let's take a look at the underlying Solr queries
>> in the log. The Streaming Expressions just creates solr queries, in this
>> case queries to the /export handler. So when something is not working as
>> expected we want to strip away the streaming expression and debug the
>> actual queries that are being run.
>>
>> You can find the Solr queries that appear in the log after running the
>> expressions and then try running them outside of the expression as plain
>> Solr queries.
>>
>> You can also post the Solr queries to this thread and we discuss what the
>> logs say.
>>
>> In these cases the logs always are the way to debug whats going on.
>>
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>>
>> On Wed, Nov 6, 2019 at 4:14 PM Jörn Franke <jo...@gmail.com> wrote:
>>
>>> I created a JIRA for this:
>>> https://issues.apache.org/jira/browse/SOLR-13894
>>>
>>> On Wed, Nov 6, 2019 at 10:45 AM Jörn Franke <jo...@gmail.com>
>>> wrote:
>>>
>>>> I have checked now Solr 8.3 server in admin UI. Same issue.
>>>>
>>>> Reproduction:
>>>> select(search(testcollection,q=“test”,df=“Default”,defType=“edismax”,fl=“id”,
>>>> qt=“/export”, sort=“id asc”),id,if(eq(1,1),Y,N) as found)
>>>>
>>>> In 8.3 it returns only the id field.
>>>> In 8.2 it returns id,found field.
>>>>
>>>> Since found is generated by select (and not coming from the collection)
>>>> there must be an issue with select.
>>>>
>>>> Any idea why this is happening.
>>>>
>>>> Debug logs do not show any error and the expression is correctly
>>>> received by Solr.
>>>>
>>>> Thank you.
>>>>
>>>> Best regards
>>>>
>>>> > Am 05.11.2019 um 14:59 schrieb Jörn Franke <jo...@gmail.com>:
>>>> >
>>>> > Thanks I will check and come back to you. As far as I remember (but
>>>> have to check) the queries generated by Solr were correct
>>>> >
>>>> > Just to be clear the same thing works with Solr 8.2 server and Solr
>>>> 8.2 client.
>>>> >
>>>> > It show the odd behaviour with Solr 8.2 server and Solr 8.3 client.
>>>> >
>>>> >> Am 05.11.2019 um 14:49 schrieb Joel Bernstein <jo...@gmail.com>:
>>>> >>
>>>> >> I'll probably need some more details. One thing that's useful is to
>>>> look at
>>>> >> the logs and see the underlying Solr queries that are generated.
>>>> Then try
>>>> >> those underlying queries against the Solr index and see what comes
>>>> back. If
>>>> >> you're not seeing the fields with the plain Solr queries then we
>>>> know it's
>>>> >> something going on below streaming expressions. If you are seeing the
>>>> >> fields then it's the expressions themselves that are not handling
>>>> the data
>>>> >> as expected.
>>>> >>
>>>> >>
>>>> >> Joel Bernstein
>>>> >> http://joelsolr.blogspot.com/
>>>> >>
>>>> >>
>>>> >>>> On Mon, Nov 4, 2019 at 9:09 AM Jörn Franke <jo...@gmail.com>
>>>> wrote:
>>>> >>>
>>>> >>> Most likely this issue can bei also reproduced in the admin UI for
>>>> the
>>>> >>> streaming handler of a collection.
>>>> >>>
>>>> >>>>> Am 04.11.2019 um 13:32 schrieb Jörn Franke <jornfranke@gmail.com
>>>> >:
>>>> >>>>
>>>> >>>> Hi,
>>>> >>>>
>>>> >>>> I use streaming expressions, e.g.
>>>> >>>> Sort(Select(search(...),id,if(eq(1,1),Y,N) as found), by=“field A
>>>> asc”)
>>>> >>>> (Using export handler, sort is not really mandatory , I will
>>>> remove it
>>>> >>> later anyway)
>>>> >>>>
>>>> >>>> This works perfectly fine if I use Solr 8.2.0 (server + client). It
>>>> >>> returns Tuples in the form { “id”,”12345”, “found”:”Y”}
>>>> >>>>
>>>> >>>> However, if I use Solr 8.2.0 as server and Solr 8.3.0 as client
>>>> then the
>>>> >>> above statement only returns the id field, but not the found field.
>>>> >>>>
>>>> >>>> Questions:
>>>> >>>> 1) is this expected behavior, ie Solr client 8.3.0 is in this case
>>>> not
>>>> >>> compatible with Solr 8.2.0 and server upgrade to Solr 8.3.0 will
>>>> fix this?
>>>> >>>> 2) has the syntax for the above expression changed? If so how?
>>>> >>>> 3) is this not expected behavior and I should create a Jira for it?
>>>> >>>>
>>>> >>>> Thank you.
>>>> >>>> Best regards
>>>> >>>
>>>>
>>>
Re: Solr 8.3 Solrj streaming expressions do not return all field values
Posted by Jörn Franke <jo...@gmail.com>.
thanks i will find another installation of Solr, but last time the
underlying queries to the export handler were correct. What was wrong was
the generated field "found" based on the if statement based NOT on any data
in the collectioj. As said, in Solr 8.3 the id fields are returned, but
not the "found" field, which is generated by the selected statement:
select(search(testcollection,q="test",df="Default",defType="edismax",fl="id",
qt="/export", sort="id asc"),id,if(eq(1,1),Y,N) as found)
returns a tuple
id: "blabla"
but not the part generated by the if(eq(1,1),Y,N) as found. In Solr 8.2 the
found field is returned:
id: "blabla"
found: "Y"
To my best knowledge this is a fully generated field => it does not depend
on the underlying collection data and the export handler. This was done to
exclude any issues with this one.
it states simply if 1=1 return Y else N
On Fri, Nov 8, 2019 at 3:20 PM Joel Bernstein <jo...@gmail.com> wrote:
> Before moving to a jira let's take a look at the underlying Solr queries
> in the log. The Streaming Expressions just creates solr queries, in this
> case queries to the /export handler. So when something is not working as
> expected we want to strip away the streaming expression and debug the
> actual queries that are being run.
>
> You can find the Solr queries that appear in the log after running the
> expressions and then try running them outside of the expression as plain
> Solr queries.
>
> You can also post the Solr queries to this thread and we discuss what the
> logs say.
>
> In these cases the logs always are the way to debug whats going on.
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Wed, Nov 6, 2019 at 4:14 PM Jörn Franke <jo...@gmail.com> wrote:
>
>> I created a JIRA for this:
>> https://issues.apache.org/jira/browse/SOLR-13894
>>
>> On Wed, Nov 6, 2019 at 10:45 AM Jörn Franke <jo...@gmail.com> wrote:
>>
>>> I have checked now Solr 8.3 server in admin UI. Same issue.
>>>
>>> Reproduction:
>>> select(search(testcollection,q=“test”,df=“Default”,defType=“edismax”,fl=“id”,
>>> qt=“/export”, sort=“id asc”),id,if(eq(1,1),Y,N) as found)
>>>
>>> In 8.3 it returns only the id field.
>>> In 8.2 it returns id,found field.
>>>
>>> Since found is generated by select (and not coming from the collection)
>>> there must be an issue with select.
>>>
>>> Any idea why this is happening.
>>>
>>> Debug logs do not show any error and the expression is correctly
>>> received by Solr.
>>>
>>> Thank you.
>>>
>>> Best regards
>>>
>>> > Am 05.11.2019 um 14:59 schrieb Jörn Franke <jo...@gmail.com>:
>>> >
>>> > Thanks I will check and come back to you. As far as I remember (but
>>> have to check) the queries generated by Solr were correct
>>> >
>>> > Just to be clear the same thing works with Solr 8.2 server and Solr
>>> 8.2 client.
>>> >
>>> > It show the odd behaviour with Solr 8.2 server and Solr 8.3 client.
>>> >
>>> >> Am 05.11.2019 um 14:49 schrieb Joel Bernstein <jo...@gmail.com>:
>>> >>
>>> >> I'll probably need some more details. One thing that's useful is to
>>> look at
>>> >> the logs and see the underlying Solr queries that are generated. Then
>>> try
>>> >> those underlying queries against the Solr index and see what comes
>>> back. If
>>> >> you're not seeing the fields with the plain Solr queries then we know
>>> it's
>>> >> something going on below streaming expressions. If you are seeing the
>>> >> fields then it's the expressions themselves that are not handling the
>>> data
>>> >> as expected.
>>> >>
>>> >>
>>> >> Joel Bernstein
>>> >> http://joelsolr.blogspot.com/
>>> >>
>>> >>
>>> >>>> On Mon, Nov 4, 2019 at 9:09 AM Jörn Franke <jo...@gmail.com>
>>> wrote:
>>> >>>
>>> >>> Most likely this issue can bei also reproduced in the admin UI for
>>> the
>>> >>> streaming handler of a collection.
>>> >>>
>>> >>>>> Am 04.11.2019 um 13:32 schrieb Jörn Franke <jo...@gmail.com>:
>>> >>>>
>>> >>>> Hi,
>>> >>>>
>>> >>>> I use streaming expressions, e.g.
>>> >>>> Sort(Select(search(...),id,if(eq(1,1),Y,N) as found), by=“field A
>>> asc”)
>>> >>>> (Using export handler, sort is not really mandatory , I will remove
>>> it
>>> >>> later anyway)
>>> >>>>
>>> >>>> This works perfectly fine if I use Solr 8.2.0 (server + client). It
>>> >>> returns Tuples in the form { “id”,”12345”, “found”:”Y”}
>>> >>>>
>>> >>>> However, if I use Solr 8.2.0 as server and Solr 8.3.0 as client
>>> then the
>>> >>> above statement only returns the id field, but not the found field.
>>> >>>>
>>> >>>> Questions:
>>> >>>> 1) is this expected behavior, ie Solr client 8.3.0 is in this case
>>> not
>>> >>> compatible with Solr 8.2.0 and server upgrade to Solr 8.3.0 will fix
>>> this?
>>> >>>> 2) has the syntax for the above expression changed? If so how?
>>> >>>> 3) is this not expected behavior and I should create a Jira for it?
>>> >>>>
>>> >>>> Thank you.
>>> >>>> Best regards
>>> >>>
>>>
>>
Re: Solr 8.3 Solrj streaming expressions do not return all field values
Posted by Joel Bernstein <jo...@gmail.com>.
Before moving to a jira let's take a look at the underlying Solr queries in
the log. The Streaming Expressions just creates solr queries, in this case
queries to the /export handler. So when something is not working as
expected we want to strip away the streaming expression and debug the
actual queries that are being run.
You can find the Solr queries that appear in the log after running the
expressions and then try running them outside of the expression as plain
Solr queries.
You can also post the Solr queries to this thread and we discuss what the
logs say.
In these cases the logs always are the way to debug whats going on.
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Nov 6, 2019 at 4:14 PM Jörn Franke <jo...@gmail.com> wrote:
> I created a JIRA for this:
> https://issues.apache.org/jira/browse/SOLR-13894
>
> On Wed, Nov 6, 2019 at 10:45 AM Jörn Franke <jo...@gmail.com> wrote:
>
>> I have checked now Solr 8.3 server in admin UI. Same issue.
>>
>> Reproduction:
>> select(search(testcollection,q=“test”,df=“Default”,defType=“edismax”,fl=“id”,
>> qt=“/export”, sort=“id asc”),id,if(eq(1,1),Y,N) as found)
>>
>> In 8.3 it returns only the id field.
>> In 8.2 it returns id,found field.
>>
>> Since found is generated by select (and not coming from the collection)
>> there must be an issue with select.
>>
>> Any idea why this is happening.
>>
>> Debug logs do not show any error and the expression is correctly received
>> by Solr.
>>
>> Thank you.
>>
>> Best regards
>>
>> > Am 05.11.2019 um 14:59 schrieb Jörn Franke <jo...@gmail.com>:
>> >
>> > Thanks I will check and come back to you. As far as I remember (but
>> have to check) the queries generated by Solr were correct
>> >
>> > Just to be clear the same thing works with Solr 8.2 server and Solr 8.2
>> client.
>> >
>> > It show the odd behaviour with Solr 8.2 server and Solr 8.3 client.
>> >
>> >> Am 05.11.2019 um 14:49 schrieb Joel Bernstein <jo...@gmail.com>:
>> >>
>> >> I'll probably need some more details. One thing that's useful is to
>> look at
>> >> the logs and see the underlying Solr queries that are generated. Then
>> try
>> >> those underlying queries against the Solr index and see what comes
>> back. If
>> >> you're not seeing the fields with the plain Solr queries then we know
>> it's
>> >> something going on below streaming expressions. If you are seeing the
>> >> fields then it's the expressions themselves that are not handling the
>> data
>> >> as expected.
>> >>
>> >>
>> >> Joel Bernstein
>> >> http://joelsolr.blogspot.com/
>> >>
>> >>
>> >>>> On Mon, Nov 4, 2019 at 9:09 AM Jörn Franke <jo...@gmail.com>
>> wrote:
>> >>>
>> >>> Most likely this issue can bei also reproduced in the admin UI for the
>> >>> streaming handler of a collection.
>> >>>
>> >>>>> Am 04.11.2019 um 13:32 schrieb Jörn Franke <jo...@gmail.com>:
>> >>>>
>> >>>> Hi,
>> >>>>
>> >>>> I use streaming expressions, e.g.
>> >>>> Sort(Select(search(...),id,if(eq(1,1),Y,N) as found), by=“field A
>> asc”)
>> >>>> (Using export handler, sort is not really mandatory , I will remove
>> it
>> >>> later anyway)
>> >>>>
>> >>>> This works perfectly fine if I use Solr 8.2.0 (server + client). It
>> >>> returns Tuples in the form { “id”,”12345”, “found”:”Y”}
>> >>>>
>> >>>> However, if I use Solr 8.2.0 as server and Solr 8.3.0 as client then
>> the
>> >>> above statement only returns the id field, but not the found field.
>> >>>>
>> >>>> Questions:
>> >>>> 1) is this expected behavior, ie Solr client 8.3.0 is in this case
>> not
>> >>> compatible with Solr 8.2.0 and server upgrade to Solr 8.3.0 will fix
>> this?
>> >>>> 2) has the syntax for the above expression changed? If so how?
>> >>>> 3) is this not expected behavior and I should create a Jira for it?
>> >>>>
>> >>>> Thank you.
>> >>>> Best regards
>> >>>
>>
>