You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Hui Liu <hl...@opentext.com> on 2016/06/05 18:11:00 UTC
Help needed on Solr Streaming Expressions
Hi,
I have Solr 6.0.0 installed on my PC (windows 7), I was experimenting with 'Streaming Expression' feature by following steps from this link: https://cwiki.apache.org/confluence/display/solr/Streaming+Expressions, but cannot get it to work, attached is my solrconfig.xml and schema.xml, note I do have 'export' handler defined in my 'solrconfig.xml' and enabled all fields as 'docvalues' in 'schema.xml'; I am using solr cloud and external zookeeper (also installed on m PC), here is the command to start this 2-node Solr cloud instance and to create the collection 'document3':
-- start 2-node solr cloud instances:
solr start -c -z 127.0.0.1:2181 -p 8988 -s solr3
solr start -c -z 127.0.0.1:2181 -p 8989 -s solr4
-- create the collection:
solr create -c document3 -d new_doc_configs3 -p 8988 -s 2 -rf 2
after creating the collection I loaded a few documents using 'csv' format and I was able to query it using 'curl' command from my PC:
-- this works on my PC:
curl http://localhost:8988/solr/document3/select?q=*:*&sort=document_id+desc,sender_msg_dest+desc&fl=document_id,sender_msg_dest,recip_msg_dest
but when trying Streaming 'search' using curl, it does not work, I tried with 3 different options: with zkHost, using 'export', or using 'select', all getting the same error:
curl: (6) Could not resolve host: sort=document_id asc,qt=
{"result-set":{"docs":[
{"EXCEPTION":null,"EOF":true}]}}
-- different curl commands tried, all getting the same error above:
curl --data-urlencode 'expr=search(document3,zkHost="127.0.0.1:2181",q="*:*",fl="document_id, sender_msg_dest", sort="document_id asc",qt="/export")' "http://localhost:8988/solr/document2/stream"
curl --data-urlencode 'expr=search(document3,q="*:*",fl="document_id, sender_msg_dest", sort="document_id asc",qt="/export")' "http://localhost:8988/solr/document2/stream"
curl --data-urlencode 'expr=search(document3,q="*:*",fl="document_id, sender_msg_dest", sort="document_id asc",qt="/select",rows=10)' "http://localhost:8988/solr/document2/stream"
what am I doing wrong? Thanks for any help!
Regards,
Hui Liu
RE: Help needed on Solr Streaming Expressions
Posted by Hui Liu <hl...@opentext.com>.
The only difference between document3 and document5 is document3 has no data in 'shard2', after loading some data into shard2, the http command also worked:
http://localhost:8988/solr/document3/stream?expr=search(document3,zkHost="127.0.0.1:2181",q="*:*",fl="document_id, sender_msg_dest", sort="document_id asc",qt="/export")
my guess is the 'null pointer' error from the stack trace is caused by no data in the 'shard2'.
Regards,
Hui
-----Original Message-----
From: Hui Liu
Sent: Monday, June 06, 2016 1:04 PM
To: solr-user@lucene.apache.org
Subject: RE: Help needed on Solr Streaming Expressions
Joel,
Thank you very much for your help, I tried the http command below with my existing 2 shards collection 'document3' (sorry I have a typo below should be document3 instead of document2), this time I got much better error:
{"result-set":{"docs":[
{"EXCEPTION":"Unable to construct instance of org.apache.solr.client.solrj.io.stream.CloudSolrStream","EOF":true}]}}
I attach the error stack trace from 'solr-8988-console.log' and 'solr.log' here in file 'solr_error.txt'.
However I continued and tried create another identical collection 'document5' with 2 shards and 2 replica using the same schema, this time the http URL worked!!! Maybe my previous collection 'document3' has some corruption?
-- command to create collection 'document5':
solr create -c document5 -d new_doc_configs5 -p 8988 -s 2 -rf 2
-- command for stream expression:
http://localhost:8988/solr/document5/stream?expr=search(document5,zkHost="127.0.0.1:2181",q="*:*",fl="document_id, sender_msg_dest", sort="document_id asc",qt="/export")
-- result from browser:
{"result-set":{"docs":[
{"document_id":20346005172,"sender_msg_dest":"ZZ:035239425"},
{"document_id":20346005173,"sender_msg_dest":"ZZ:035239425"},
{"document_id":20346006403,"sender_msg_dest":"14:004321519IBMP"},
{"document_id":20346006406,"sender_msg_dest":"14:004321519IBMP"},
{"document_id":20346006741,"sender_msg_dest":"14:004321519IBMP"},
{"document_id":20346006743,"sender_msg_dest":"14:004321519IBMP"},
{"EOF":true,"RESPONSE_TIME":10}]}}
Do you think I can try the same in http using other 'Stream Decorators' such as 'complement' and 'innerJoin'?
Regards,
Hui
-----Original Message-----
From: Joel Bernstein [mailto:joelsolr@gmail.com]
Sent: Monday, June 06, 2016 9:51 AM
To: solr-user@lucene.apache.org
Subject: Re: Help needed on Solr Streaming Expressions
Hi,
To eliminate any issues that might be happening due to curl, try running the command from your browser.
http://localhost:8988/solr/document2/stream?expr=search(document3,zkHost="
127.0.0.1:2181",q="*:*",fl="document_id, sender_msg_dest", sort="document_id asc",qt="/export")
I think most browsers will url encode the expression automatically, but you can url encode also using an online tool. Also you can remove the zkHost param and it should default to zkHost your solr is connected to.
If you still get an error take a look at the logs and post the full stack trace to this thread, which will help determine where the problem is.
Joel Bernstein
http://joelsolr.blogspot.com/
On Sun, Jun 5, 2016 at 2:11 PM, Hui Liu <hl...@opentext.com> wrote:
> Hi,
>
>
>
> I have Solr 6.0.0 installed on my PC (windows 7), I was
> experimenting with ‘Streaming Expression’ feature by following steps
> from this link:
> https://cwiki.apache.org/confluence/display/solr/Streaming+Expressions
> , but cannot get it to work, attached is my solrconfig.xml and
> schema.xml, note I do have ‘export’ handler defined in my
> ‘solrconfig.xml’ and enabled all fields as ‘docvalues’ in
> ‘schema.xml’; I am using solr cloud and external zookeeper (also
> installed on m PC), here is the command to start this 2-node Solr
> cloud instance and to create the collection ‘document3’:
>
>
>
> -- start 2-node solr cloud instances:
>
> solr start -c -z 127.0.0.1:2181 -p 8988 -s solr3
>
> solr start -c -z 127.0.0.1:2181 -p 8989 -s solr4
>
>
>
> -- create the collection:
>
> solr create -c document3 -d new_doc_configs3 -p 8988 -s 2 -rf 2
>
>
>
> after creating the collection I loaded a few documents
> using ‘csv’ format and I was able to query it using ‘curl’ command from my PC:
>
>
>
> -- this works on my PC:
>
> curl
> http://localhost:8988/solr/document3/select?q=*:*&sort=document_id+des
> c,sender_msg_dest+desc&fl=document_id,sender_msg_dest,recip_msg_dest
>
>
>
> but when trying Streaming ‘search’ using curl, it does
> not work, I tried with 3 different options: with zkHost, using
> ‘export’, or using ‘select’, all getting the same error:
>
>
> curl: (6) Could not resolve host: sort=document_id asc,qt=
>
> {"result-set":{"docs":[
>
> {"EXCEPTION":null,"EOF":true}]}}
>
> -- different curl commands tried, all getting the same error above:
>
> curl --data-urlencode
> 'expr=search(document3,zkHost="127.0.0.1:2181",q="*:*",fl="document_id
> , sender_msg_dest", sort="document_id asc",qt="/export")' "
> http://localhost:8988/solr/document2/stream"
>
>
>
> curl --data-urlencode 'expr=search(document3,q="*:*",fl="document_id,
> sender_msg_dest", sort="document_id asc",qt="/export")' "
> http://localhost:8988/solr/document2/stream"
>
>
>
> curl --data-urlencode 'expr=search(document3,q="*:*",fl="document_id,
> sender_msg_dest", sort="document_id asc",qt="/select",rows=10)' "
> http://localhost:8988/solr/document2/stream"
>
>
>
> what am I doing wrong? Thanks for any help!
>
>
>
> Regards,
>
> Hui Liu
>
RE: Help needed on Solr Streaming Expressions
Posted by Hui Liu <hl...@opentext.com>.
Joel,
Thank you very much for your help, I tried the http command below with my existing 2 shards collection 'document3' (sorry I have a typo below should be document3 instead of document2), this time I got much better error:
{"result-set":{"docs":[
{"EXCEPTION":"Unable to construct instance of org.apache.solr.client.solrj.io.stream.CloudSolrStream","EOF":true}]}}
I attach the error stack trace from 'solr-8988-console.log' and 'solr.log' here in file 'solr_error.txt'.
However I continued and tried create another identical collection 'document5' with 2 shards and 2 replica using the same schema, this time the http URL worked!!! Maybe my previous collection 'document3' has some corruption?
-- command to create collection 'document5':
solr create -c document5 -d new_doc_configs5 -p 8988 -s 2 -rf 2
-- command for stream expression:
http://localhost:8988/solr/document5/stream?expr=search(document5,zkHost="127.0.0.1:2181",q="*:*",fl="document_id, sender_msg_dest", sort="document_id asc",qt="/export")
-- result from browser:
{"result-set":{"docs":[
{"document_id":20346005172,"sender_msg_dest":"ZZ:035239425"},
{"document_id":20346005173,"sender_msg_dest":"ZZ:035239425"},
{"document_id":20346006403,"sender_msg_dest":"14:004321519IBMP"},
{"document_id":20346006406,"sender_msg_dest":"14:004321519IBMP"},
{"document_id":20346006741,"sender_msg_dest":"14:004321519IBMP"},
{"document_id":20346006743,"sender_msg_dest":"14:004321519IBMP"},
{"EOF":true,"RESPONSE_TIME":10}]}}
Do you think I can try the same in http using other 'Stream Decorators' such as 'complement' and 'innerJoin'?
Regards,
Hui
-----Original Message-----
From: Joel Bernstein [mailto:joelsolr@gmail.com]
Sent: Monday, June 06, 2016 9:51 AM
To: solr-user@lucene.apache.org
Subject: Re: Help needed on Solr Streaming Expressions
Hi,
To eliminate any issues that might be happening due to curl, try running the command from your browser.
http://localhost:8988/solr/document2/stream?expr=search(document3,zkHost="
127.0.0.1:2181",q="*:*",fl="document_id, sender_msg_dest", sort="document_id asc",qt="/export")
I think most browsers will url encode the expression automatically, but you can url encode also using an online tool. Also you can remove the zkHost param and it should default to zkHost your solr is connected to.
If you still get an error take a look at the logs and post the full stack trace to this thread, which will help determine where the problem is.
Joel Bernstein
http://joelsolr.blogspot.com/
On Sun, Jun 5, 2016 at 2:11 PM, Hui Liu <hl...@opentext.com> wrote:
> Hi,
>
>
>
> I have Solr 6.0.0 installed on my PC (windows 7), I was
> experimenting with ‘Streaming Expression’ feature by following steps
> from this link:
> https://cwiki.apache.org/confluence/display/solr/Streaming+Expressions
> , but cannot get it to work, attached is my solrconfig.xml and
> schema.xml, note I do have ‘export’ handler defined in my
> ‘solrconfig.xml’ and enabled all fields as ‘docvalues’ in
> ‘schema.xml’; I am using solr cloud and external zookeeper (also
> installed on m PC), here is the command to start this 2-node Solr
> cloud instance and to create the collection ‘document3’:
>
>
>
> -- start 2-node solr cloud instances:
>
> solr start -c -z 127.0.0.1:2181 -p 8988 -s solr3
>
> solr start -c -z 127.0.0.1:2181 -p 8989 -s solr4
>
>
>
> -- create the collection:
>
> solr create -c document3 -d new_doc_configs3 -p 8988 -s 2 -rf 2
>
>
>
> after creating the collection I loaded a few documents
> using ‘csv’ format and I was able to query it using ‘curl’ command from my PC:
>
>
>
> -- this works on my PC:
>
> curl
> http://localhost:8988/solr/document3/select?q=*:*&sort=document_id+des
> c,sender_msg_dest+desc&fl=document_id,sender_msg_dest,recip_msg_dest
>
>
>
> but when trying Streaming ‘search’ using curl, it does
> not work, I tried with 3 different options: with zkHost, using
> ‘export’, or using ‘select’, all getting the same error:
>
>
> curl: (6) Could not resolve host: sort=document_id asc,qt=
>
> {"result-set":{"docs":[
>
> {"EXCEPTION":null,"EOF":true}]}}
>
> -- different curl commands tried, all getting the same error above:
>
> curl --data-urlencode
> 'expr=search(document3,zkHost="127.0.0.1:2181",q="*:*",fl="document_id
> , sender_msg_dest", sort="document_id asc",qt="/export")' "
> http://localhost:8988/solr/document2/stream"
>
>
>
> curl --data-urlencode 'expr=search(document3,q="*:*",fl="document_id,
> sender_msg_dest", sort="document_id asc",qt="/export")' "
> http://localhost:8988/solr/document2/stream"
>
>
>
> curl --data-urlencode 'expr=search(document3,q="*:*",fl="document_id,
> sender_msg_dest", sort="document_id asc",qt="/select",rows=10)' "
> http://localhost:8988/solr/document2/stream"
>
>
>
> what am I doing wrong? Thanks for any help!
>
>
>
> Regards,
>
> Hui Liu
>
Re: Help needed on Solr Streaming Expressions
Posted by Joel Bernstein <jo...@gmail.com>.
Hi,
To eliminate any issues that might be happening due to curl, try running
the command from your browser.
http://localhost:8988/solr/document2/stream?expr=search(document3,zkHost="
127.0.0.1:2181",q="*:*",fl="document_id, sender_msg_dest",
sort="document_id asc",qt="/export")
I think most browsers will url encode the expression automatically, but you
can url encode also using an online tool. Also you can remove the zkHost
param and it should default to zkHost your solr is connected to.
If you still get an error take a look at the logs and post the full stack
trace to this thread, which will help determine where the problem is.
Joel Bernstein
http://joelsolr.blogspot.com/
On Sun, Jun 5, 2016 at 2:11 PM, Hui Liu <hl...@opentext.com> wrote:
> Hi,
>
>
>
> I have Solr 6.0.0 installed on my PC (windows 7), I was
> experimenting with ‘Streaming Expression’ feature by following steps from
> this link:
> https://cwiki.apache.org/confluence/display/solr/Streaming+Expressions,
> but cannot get it to work, attached is my solrconfig.xml and schema.xml,
> note I do have ‘export’ handler defined in my ‘solrconfig.xml’ and enabled
> all fields as ‘docvalues’ in ‘schema.xml’; I am using solr cloud and
> external zookeeper (also installed on m PC), here is the command to start
> this 2-node Solr cloud instance and to create the collection ‘document3’:
>
>
>
> -- start 2-node solr cloud instances:
>
> solr start -c -z 127.0.0.1:2181 -p 8988 -s solr3
>
> solr start -c -z 127.0.0.1:2181 -p 8989 -s solr4
>
>
>
> -- create the collection:
>
> solr create -c document3 -d new_doc_configs3 -p 8988 -s 2 -rf 2
>
>
>
> after creating the collection I loaded a few documents using
> ‘csv’ format and I was able to query it using ‘curl’ command from my PC:
>
>
>
> -- this works on my PC:
>
> curl
> http://localhost:8988/solr/document3/select?q=*:*&sort=document_id+desc,sender_msg_dest+desc&fl=document_id,sender_msg_dest,recip_msg_dest
>
>
>
> but when trying Streaming ‘search’ using curl, it does not
> work, I tried with 3 different options: with zkHost, using ‘export’, or
> using ‘select’, all getting the same error:
>
>
> curl: (6) Could not resolve host: sort=document_id asc,qt=
>
> {"result-set":{"docs":[
>
> {"EXCEPTION":null,"EOF":true}]}}
>
> -- different curl commands tried, all getting the same error above:
>
> curl --data-urlencode 'expr=search(document3,zkHost="127.0.0.1:2181",q="*:*",fl="document_id,
> sender_msg_dest", sort="document_id asc",qt="/export")' "
> http://localhost:8988/solr/document2/stream"
>
>
>
> curl --data-urlencode 'expr=search(document3,q="*:*",fl="document_id,
> sender_msg_dest", sort="document_id asc",qt="/export")' "
> http://localhost:8988/solr/document2/stream"
>
>
>
> curl --data-urlencode 'expr=search(document3,q="*:*",fl="document_id,
> sender_msg_dest", sort="document_id asc",qt="/select",rows=10)' "
> http://localhost:8988/solr/document2/stream"
>
>
>
> what am I doing wrong? Thanks for any help!
>
>
>
> Regards,
>
> Hui Liu
>