You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Adarsh_infor <hd...@infor.com> on 2018/06/28 06:01:49 UTC
A user defined request handler is failing to fetch the data.
Hi All,
I was running SOLR 4.6.1 in master slave architecture on Windows machine,
Now am planning to migrate that to Linux and upgrade the SOLR 6.6.3
version(Remember only the Configs and schema will be migtrated not data). In
the process of doing so i had to do some changes to Schema and Config in
order to create a core without an issue, after creating the core i was
getting a warning like the LUCENE_40 support will be removed from 7 so need
to change it. So in order to fix that warning i have changed it
`<luceneMatchVersion>6.6.3</luceneMatchVersion>`. After which using *DIH*
i indexed certain documents and was able to query them by using default
`/select` request handler. But i wanted to test the distributed search so in
order to test that . i have created one more core in the same server, which
is having a below requestHandler defined in config.xml
<requestHandler name="/filesearch" class="solr.SearchHandler">
<lst name="defaults">
<str name="q.alt">*:*</str>
<str name="shards">localhost:8983/solr/FI_idx</str>
<str name="df">document</str>
</lst>
</requestHandler>
Core *FI_idx* is the core which is having the indexed data in it, with
`<luceneMatchVersion>6.6.3</luceneMatchVersion>` but when query using
requestHandler `/filesearch` and getting below error could anyone help to me
to understand whats happening.
"trace":
"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
from server at http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx: Error from server at
http://localhost:8983/solr/FI_idx:
org.apache.solr.client.solrj.SolrServerException: IOException occured when
talking to server at: http://localhost:8983/solr/FI_idx\n\tat
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)\n\tat
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)\n\tat
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)\n\tat
org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)\n\tat
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:218)\n\tat
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:183)\n\tat
java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat
java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\tat
java.lang.Thread.run(Thread.java:745)\n",
"code": 500
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: A user defined request handler is failing to fetch the data.
Posted by Adarsh_infor <hd...@infor.com>.
Hi Shawn,
thanks that helped. I modified the searchHandler as below and it started
working
<requestHandler name="/filesearch" class="solr.SearchHandler">
<lst name="defaults">
<str name="q.alt">*:*</str>
<str name="shards">localhost:8983/solr/FI_idx</str>
<str name="shards.qt">/select</str>
</lst>
</requestHandler>
Regards
Adarsh
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: A user defined request handler is failing to fetch the data.
Posted by Shawn Heisey <ap...@elyograg.org>.
On 7/3/2018 3:09 AM, Adarsh_infor wrote:
> the below command works .
>
> http://localhost:8983/solr/FI_idx/select?q=*:*&indent=true&shards=localhost:8983/solr/FI_idx
>
> but the same will not work with filesearch new search handler.
>
> http://localhost:8983/solr/FI_idx/filesearch?q=*:*&indent=true&shards=localhost:8983/solr/FI_idx
>
> This gives me error, so am trying to figure what difference could have
> caused the failed for the second http command.
>
> And the second http command works fine if i change the lucenematchversion in
> solrconfig to LUCENE_40 which is totally weird because the indexing has
> happened with lucenematchversion 6.6.3 but search works fine with LUCENE_40.
> which is totally weird behaviour.
I think I know what's going on here.
The reason that 4.0 compatibility works is this issue that takes effect
without it:
https://issues.apache.org/jira/browse/SOLR-6311
What this issue does internally is turn your second URL into this (split
into two lines for readability):
http://localhost:8983/solr/FI_idx/filesearch?q=*:*&indent=true
&shards=localhost:8983/solr/FI_idx&shards.qt=/filesearch
Before SOLR-6311 (Solr 5.0 and earlier), shard subrequests would always
go to /select. Starting with version 5.1, shard subrequests go to the
same handler that received the request.
I bet your /select handler does NOT have a definition for the shards
parameter. So when issuing the first request, or the second request
with 4.0 compatibility, the initial request has the shards parameter,
the subrequest goes to the /select handler, and everything's good.
With the second request and the 6.x version behavior, you make an
initial request, and include a shards parameter. The subrequest made by
the shards parameter you included on the URL goes to the same handler
(/filesearch), which has a shards parameter in its definition. That
shards parameter sends the request AGAIN to /filesearch. And again.
And again. This repeats over and over until the java virtual machine
runs out of some resource (might be heap, might be stack space, might be
something else) and can't follow the rabbit hole down any more.
If you were to include the following parameter on the second request, it
should work:
&shards.qt=/select
Thanks,
Shawn
Re: A user defined request handler is failing to fetch the data.
Posted by Adarsh_infor <hd...@infor.com>.
@Erick @Shawn
Adding to my previous comment.
the below command works .
http://localhost:8983/solr/FI_idx/select?q=*:*&indent=true&shards=localhost:8983/solr/FI_idx
but the same will not work with filesearch new search handler.
http://localhost:8983/solr/FI_idx/filesearch?q=*:*&indent=true&shards=localhost:8983/solr/FI_idx
This gives me error, so am trying to figure what difference could have
caused the failed for the second http command.
And the second http command works fine if i change the lucenematchversion in
solrconfig to LUCENE_40 which is totally weird because the indexing has
happened with lucenematchversion 6.6.3 but search works fine with LUCENE_40.
which is totally weird behaviour.
Thanks
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: A user defined request handler is failing to fetch the data.
Posted by Adarsh_infor <hd...@infor.com>.
@Shawn Heisey-2
When we say recursive shards, what does that mean? My distributed node will
not have any data in it it will be just used for searching all the
shards(nodes) where the documents are indexed and try to get consolidated
data from it. My only problem here is if i change the <luceneMatchVersion>
to LUCENE_40 everything seems to be working fine, but if we change that to
6.6.3 or LUCENE_CURRENT it starts breaking so does that meaning the
distributed search is not supported from Lucene 6.* version ?
Thanks
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: A user defined request handler is failing to fetch the data.
Posted by Shawn Heisey <ap...@elyograg.org>.
On 7/2/2018 12:58 AM, Adarsh_infor wrote:
> Yes am going to have the shards on 6 different servers which will be later
> called in my searchHandler by specifying the shards list. But for that
> initially i was testing the filesearch with the single shard which was
> suppose to work. I know solr could does handle these thing more better than
> but for now i need to use the master/slave architecture with distributed
> node in front of them. As of now if in the solrconfig.xml if i keep
> lucenematchversion to 6.6.3 then only . am seeing the error which i posted
> earlier if switch the version back to LUCENE_40 it just works fine. But is
> it not suppose to work with 6.6.3 am confused there.
>
> And also the logs which i pasted in from solr.log not from the client side.
I agree with Erick that this looks like a problem with a recursive
shards parameter. I set up a test handler with a recursive shards
parameter, and when I tried a request to that handler, it spit out an
error message that looked just like what you're seeing.
It is normally a bad idea to include the shards parameter in a handler
definition. It *can* be safe, but takes a lot of special care to make
sure that recursive shard calls do not happen.
I can't imagine how setting luceneMatchVersion could alter this
behavior. Which might be a failure of imagination on my part.
Thanks,
Shawn
Re: A user defined request handler is failing to fetch the data.
Posted by Adarsh_infor <hd...@infor.com>.
@Erick Erickson
Thanks for the response.
Yes am going to have the shards on 6 different servers which will be later
called in my searchHandler by specifying the shards list. But for that
initially i was testing the filesearch with the single shard which was
suppose to work. I know solr could does handle these thing more better than
but for now i need to use the master/slave architecture with distributed
node in front of them. As of now if in the solrconfig.xml if i keep
lucenematchversion to 6.6.3 then only . am seeing the error which i posted
earlier if switch the version back to LUCENE_40 it just works fine. But is
it not suppose to work with 6.6.3 am confused there.
And also the logs which i pasted in from solr.log not from the client side.
Thanks
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: A user defined request handler is failing to fetch the data.
Posted by Erick Erickson <er...@gmail.com>.
At a glance, you may really be seeing recursive calls. What does the
solr log (rather than the client) say is happening?
Any call that comes into /filesearch will make a call to...
./filesearch perhaps?
What is your goal here? You say you're "testing distributed search".
But what do you mean by that? There are two
bits here:
1> you really mean the entire index is contained in a single core and
you want multiple cores to handle increased QPS. This isn't done by
messing with the shards parameter, since each core is a complete
index. People usually just put a load balancer in front of the slaves
(if using master/slave) or just use SolrCloud with a number of
replicas and let Solr do the work.
2> You want to use a sharded solution because you have too much data
to be handled by a single core. In that case I would _strongly_
recommend you just use SolrCloud. That would "just work" without this
kind of tweaking.
Best,
Erick
On Wed, Jun 27, 2018 at 11:01 PM, Adarsh_infor <hd...@infor.com> wrote:
> Hi All,
>
> I was running SOLR 4.6.1 in master slave architecture on Windows machine,
> Now am planning to migrate that to Linux and upgrade the SOLR 6.6.3
> version(Remember only the Configs and schema will be migtrated not data). In
> the process of doing so i had to do some changes to Schema and Config in
> order to create a core without an issue, after creating the core i was
> getting a warning like the LUCENE_40 support will be removed from 7 so need
> to change it. So in order to fix that warning i have changed it
> `<luceneMatchVersion>6.6.3</luceneMatchVersion>`. After which using *DIH*
> i indexed certain documents and was able to query them by using default
> `/select` request handler. But i wanted to test the distributed search so in
> order to test that . i have created one more core in the same server, which
> is having a below requestHandler defined in config.xml
>
> <requestHandler name="/filesearch" class="solr.SearchHandler">
> <lst name="defaults">
> <str name="q.alt">*:*</str>
> <str name="shards">localhost:8983/solr/FI_idx</str>
> <str name="df">document</str>
> </lst>
> </requestHandler>
>
> Core *FI_idx* is the core which is having the indexed data in it, with
> `<luceneMatchVersion>6.6.3</luceneMatchVersion>` but when query using
> requestHandler `/filesearch` and getting below error could anyone help to me
> to understand whats happening.
>
>
>
> "trace":
> "org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
> from server at http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx: Error from server at
> http://localhost:8983/solr/FI_idx:
> org.apache.solr.client.solrj.SolrServerException: IOException occured when
> talking to server at: http://localhost:8983/solr/FI_idx\n\tat
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)\n\tat
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)\n\tat
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)\n\tat
> org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)\n\tat
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:218)\n\tat
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:183)\n\tat
> java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat
> java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148)\n\tat
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\tat
> java.lang.Thread.run(Thread.java:745)\n",
> "code": 500
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html