You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Salman Ansari <sa...@gmail.com> on 2016/01/31 15:20:19 UTC
URI is too long
Hi,
I am building a long query containing multiple ORs between query terms. I
started to receive the following exception:
The remote server returned an error: (414) Request-URI Too Long. Any idea
what is the limit of the URL in Solr? Moreover, as a solution I was
thinking of chunking the query into multiple requests but I was wondering
if anyone has a better approach?
Regards,
Salman
Re: URI is too long
Posted by vlspavan <vl...@gmail.com>.
Hi Salman,
I have the same issue that URI is too long. Did you try implementing the
post method ? If so, can you please post the code from your end on post
requests. This will help greatly.
Thank you
Pavan
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: URI is too long
Posted by Salman Ansari <sa...@gmail.com>.
I tried using POST but faced an issue where I am still not able to send
long data. When I send data in the body that exceeds 35KB I get the
following exception:
"An exception of type 'SolrNet.Exceptions.SolrConnectionException' occurred
in [Myproject] but was not handled in user code
Additional information: The request was aborted: The connection was closed
unexpectedly."
Any ideas why this is happening and how to resolve this?
Regards,
Salman
On Mon, Feb 1, 2016 at 2:15 PM, Upayavira <uv...@odoko.co.uk> wrote:
> POST is supposed (as defined by REST) to imply a request with
> side-effects. A query as such does not have side effects, so
> conceptually, it should be a GET. In practice, whilst it might cause
> some developers to grumble, using a POST for a request should make no
> difference to Solr (other than accepting a larger query).
>
> Upayavira
>
> On Mon, Feb 1, 2016, at 11:05 AM, Midas A wrote:
> > Is there any drawback of POST request and why we prefer GET.
> >
> > On Mon, Feb 1, 2016 at 1:08 PM, Salman Ansari <sa...@gmail.com>
> > wrote:
> >
> > > Cool. I would give POST a try. Any samples of using Post while passing
> the
> > > query string values (such as ORing between Solr field values) using
> > > Solr.NET?
> > >
> > > Regards,
> > > Salman
> > >
> > > On Sun, Jan 31, 2016 at 10:21 PM, Shawn Heisey <ap...@elyograg.org>
> > > wrote:
> > >
> > > > On 1/31/2016 7:20 AM, Salman Ansari wrote:
> > > > > I am building a long query containing multiple ORs between query
> > > terms. I
> > > > > started to receive the following exception:
> > > > >
> > > > > The remote server returned an error: (414) Request-URI Too Long.
> Any
> > > idea
> > > > > what is the limit of the URL in Solr? Moreover, as a solution I was
> > > > > thinking of chunking the query into multiple requests but I was
> > > wondering
> > > > > if anyone has a better approach?
> > > >
> > > > The default HTTP header size limit on most webservers and containers
> > > > (including the Jetty that ships with Solr) is 8192 bytes. A typical
> > > > request like this will start with "GET " and end with " HTTP/1.1",
> which
> > > > count against that 8192 bytes. The max header size can be increased.
> > > >
> > > > If you place the parameters into a POST request instead of on the
> URL,
> > > > then the default size limit of that POST request in Solr is 2MB.
> This
> > > > can also be increased.
> > > >
> > > > Thanks,
> > > > Shawn
> > > >
> > > >
> > >
>
Re: URI is too long
Posted by Upayavira <uv...@odoko.co.uk>.
POST is supposed (as defined by REST) to imply a request with
side-effects. A query as such does not have side effects, so
conceptually, it should be a GET. In practice, whilst it might cause
some developers to grumble, using a POST for a request should make no
difference to Solr (other than accepting a larger query).
Upayavira
On Mon, Feb 1, 2016, at 11:05 AM, Midas A wrote:
> Is there any drawback of POST request and why we prefer GET.
>
> On Mon, Feb 1, 2016 at 1:08 PM, Salman Ansari <sa...@gmail.com>
> wrote:
>
> > Cool. I would give POST a try. Any samples of using Post while passing the
> > query string values (such as ORing between Solr field values) using
> > Solr.NET?
> >
> > Regards,
> > Salman
> >
> > On Sun, Jan 31, 2016 at 10:21 PM, Shawn Heisey <ap...@elyograg.org>
> > wrote:
> >
> > > On 1/31/2016 7:20 AM, Salman Ansari wrote:
> > > > I am building a long query containing multiple ORs between query
> > terms. I
> > > > started to receive the following exception:
> > > >
> > > > The remote server returned an error: (414) Request-URI Too Long. Any
> > idea
> > > > what is the limit of the URL in Solr? Moreover, as a solution I was
> > > > thinking of chunking the query into multiple requests but I was
> > wondering
> > > > if anyone has a better approach?
> > >
> > > The default HTTP header size limit on most webservers and containers
> > > (including the Jetty that ships with Solr) is 8192 bytes. A typical
> > > request like this will start with "GET " and end with " HTTP/1.1", which
> > > count against that 8192 bytes. The max header size can be increased.
> > >
> > > If you place the parameters into a POST request instead of on the URL,
> > > then the default size limit of that POST request in Solr is 2MB. This
> > > can also be increased.
> > >
> > > Thanks,
> > > Shawn
> > >
> > >
> >
Re: URI is too long
Posted by Midas A <te...@gmail.com>.
Is there any drawback of POST request and why we prefer GET.
On Mon, Feb 1, 2016 at 1:08 PM, Salman Ansari <sa...@gmail.com>
wrote:
> Cool. I would give POST a try. Any samples of using Post while passing the
> query string values (such as ORing between Solr field values) using
> Solr.NET?
>
> Regards,
> Salman
>
> On Sun, Jan 31, 2016 at 10:21 PM, Shawn Heisey <ap...@elyograg.org>
> wrote:
>
> > On 1/31/2016 7:20 AM, Salman Ansari wrote:
> > > I am building a long query containing multiple ORs between query
> terms. I
> > > started to receive the following exception:
> > >
> > > The remote server returned an error: (414) Request-URI Too Long. Any
> idea
> > > what is the limit of the URL in Solr? Moreover, as a solution I was
> > > thinking of chunking the query into multiple requests but I was
> wondering
> > > if anyone has a better approach?
> >
> > The default HTTP header size limit on most webservers and containers
> > (including the Jetty that ships with Solr) is 8192 bytes. A typical
> > request like this will start with "GET " and end with " HTTP/1.1", which
> > count against that 8192 bytes. The max header size can be increased.
> >
> > If you place the parameters into a POST request instead of on the URL,
> > then the default size limit of that POST request in Solr is 2MB. This
> > can also be increased.
> >
> > Thanks,
> > Shawn
> >
> >
>
Re: URI is too long
Posted by Shawn Heisey <ap...@elyograg.org>.
On 2/6/2016 2:47 PM, Maciej Lisiewski wrote:
> GET request length is limited by URL length. RFCs defining HTTP recommend
> you keep it under 8KB but put no hard limit in place.
> Because of no hard limit it boils down to the implementation - both on the
> server side and client side:
> - On client side Internet Explorer is limited to about 2KB for example
> - On server side Jetty (used by Solr) defaults to a 4KB limit
>
> POST should be used for larger requests, but it can sometimes be limited
> too - for example Jetty (default container for Solr) defaults to 200.000
> bytes max.
> To increase that change this:
> http://www.eclipse.org/jetty/documentation/current/setting-form-size.html
Solr controls the max size of the POST request. It defaults to 2MB. It
is changed with the formdataUploadLimitInKB setting in solrconfig.xml.
Changing the size limit in the container config will likely have no effect.
Thanks,
Shawn
Re: URI is too long
Posted by Maciej Lisiewski <ma...@gmail.com>.
GET request length is limited by URL length. RFCs defining HTTP recommend
you keep it under 8KB but put no hard limit in place.
Because of no hard limit it boils down to the implementation - both on the
server side and client side:
- On client side Internet Explorer is limited to about 2KB for example
- On server side Jetty (used by Solr) defaults to a 4KB limit
POST should be used for larger requests, but it can sometimes be limited
too - for example Jetty (default container for Solr) defaults to 200.000
bytes max.
To increase that change this:
http://www.eclipse.org/jetty/documentation/current/setting-form-size.html
--
Maciej Lisiewski
+1 647-779-1788
On Sat, Feb 6, 2016 at 3:11 PM, Jack Krupansky <ja...@gmail.com>
wrote:
> And you're sure that you can't use the terms query parser, which was
> explicitly designed for handling a very long list of terms to be implicitly
> ORed?
>
> -- Jack Krupansky
>
> On Sat, Feb 6, 2016 at 2:26 PM, Salman Ansari <sa...@gmail.com>
> wrote:
>
> > It looked like there was another issue with my query. I had too many
> > boolean operators (I believe maxBooleanClause property in
> SolrConfig.xml).
> > I just looped in batch of 1000 to get all the docs. Not sure if there is
> a
> > better way of handling this.
> >
> > Regards,
> > Salman
> >
> >
> > On Wed, Feb 3, 2016 at 12:29 AM, Shawn Heisey <ap...@elyograg.org>
> wrote:
> >
> > > On 2/2/2016 1:46 PM, Salman Ansari wrote:
> > > > OK then, if there is no way around this problem, can someone tell me
> > the
> > > > maximum size a POST body can handle in Solr?
> > >
> > > It is configurable in solrconfig.xml. Look for the
> > > formdataUploadLimitInKB setting in the 5.x configsets. This setting
> > > defaults to 2048, which means 2 megabytes.
> > >
> > > Thanks,
> > > Shawn
> > >
> > >
> >
>
Re: URI is too long
Posted by Jack Krupansky <ja...@gmail.com>.
And you're sure that you can't use the terms query parser, which was
explicitly designed for handling a very long list of terms to be implicitly
ORed?
-- Jack Krupansky
On Sat, Feb 6, 2016 at 2:26 PM, Salman Ansari <sa...@gmail.com>
wrote:
> It looked like there was another issue with my query. I had too many
> boolean operators (I believe maxBooleanClause property in SolrConfig.xml).
> I just looped in batch of 1000 to get all the docs. Not sure if there is a
> better way of handling this.
>
> Regards,
> Salman
>
>
> On Wed, Feb 3, 2016 at 12:29 AM, Shawn Heisey <ap...@elyograg.org> wrote:
>
> > On 2/2/2016 1:46 PM, Salman Ansari wrote:
> > > OK then, if there is no way around this problem, can someone tell me
> the
> > > maximum size a POST body can handle in Solr?
> >
> > It is configurable in solrconfig.xml. Look for the
> > formdataUploadLimitInKB setting in the 5.x configsets. This setting
> > defaults to 2048, which means 2 megabytes.
> >
> > Thanks,
> > Shawn
> >
> >
>
Re: URI is too long
Posted by Salman Ansari <sa...@gmail.com>.
It looked like there was another issue with my query. I had too many
boolean operators (I believe maxBooleanClause property in SolrConfig.xml).
I just looped in batch of 1000 to get all the docs. Not sure if there is a
better way of handling this.
Regards,
Salman
On Wed, Feb 3, 2016 at 12:29 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 2/2/2016 1:46 PM, Salman Ansari wrote:
> > OK then, if there is no way around this problem, can someone tell me the
> > maximum size a POST body can handle in Solr?
>
> It is configurable in solrconfig.xml. Look for the
> formdataUploadLimitInKB setting in the 5.x configsets. This setting
> defaults to 2048, which means 2 megabytes.
>
> Thanks,
> Shawn
>
>
Re: URI is too long
Posted by Shawn Heisey <ap...@elyograg.org>.
On 2/2/2016 1:46 PM, Salman Ansari wrote:
> OK then, if there is no way around this problem, can someone tell me the
> maximum size a POST body can handle in Solr?
It is configurable in solrconfig.xml. Look for the
formdataUploadLimitInKB setting in the 5.x configsets. This setting
defaults to 2048, which means 2 megabytes.
Thanks,
Shawn
Re: URI is too long
Posted by Salman Ansari <sa...@gmail.com>.
OK then, if there is no way around this problem, can someone tell me the
maximum size a POST body can handle in Solr?
Regards,
Salman
On Tue, Feb 2, 2016 at 12:12 AM, Salman Ansari <sa...@gmail.com>
wrote:
> That is what I have tried. I tried using POST with
> application/x-www-form-urlencoded and I got the exception I mentioned. Is
> there a way I can get around this exception?
>
> Regards,
> Salman
>
> On Mon, Feb 1, 2016 at 6:08 PM, Susheel Kumar <su...@gmail.com>
> wrote:
>
>> Post is pretty much similar to GET. You can use any REST Client to try.
>> Same select URL & pass below header and put the queries parameters into
>> body
>>
>> POST: http://localhost:8983/solr/techproducts/select
>>
>> Header
>> ==
>> Content-Type:application/x-www-form-urlencoded
>>
>> payload/body:
>> ==
>> q=*:*&rows=2
>>
>>
>> Thanks,
>> Susheel
>>
>> On Mon, Feb 1, 2016 at 2:38 AM, Salman Ansari <sa...@gmail.com>
>> wrote:
>>
>> > Cool. I would give POST a try. Any samples of using Post while passing
>> the
>> > query string values (such as ORing between Solr field values) using
>> > Solr.NET?
>> >
>> > Regards,
>> > Salman
>> >
>> > On Sun, Jan 31, 2016 at 10:21 PM, Shawn Heisey <ap...@elyograg.org>
>> > wrote:
>> >
>> > > On 1/31/2016 7:20 AM, Salman Ansari wrote:
>> > > > I am building a long query containing multiple ORs between query
>> > terms. I
>> > > > started to receive the following exception:
>> > > >
>> > > > The remote server returned an error: (414) Request-URI Too Long. Any
>> > idea
>> > > > what is the limit of the URL in Solr? Moreover, as a solution I was
>> > > > thinking of chunking the query into multiple requests but I was
>> > wondering
>> > > > if anyone has a better approach?
>> > >
>> > > The default HTTP header size limit on most webservers and containers
>> > > (including the Jetty that ships with Solr) is 8192 bytes. A typical
>> > > request like this will start with "GET " and end with " HTTP/1.1",
>> which
>> > > count against that 8192 bytes. The max header size can be increased.
>> > >
>> > > If you place the parameters into a POST request instead of on the URL,
>> > > then the default size limit of that POST request in Solr is 2MB. This
>> > > can also be increased.
>> > >
>> > > Thanks,
>> > > Shawn
>> > >
>> > >
>> >
>>
>
>
Re: URI is too long
Posted by Salman Ansari <sa...@gmail.com>.
That is what I have tried. I tried using POST with
application/x-www-form-urlencoded and I got the exception I mentioned. Is
there a way I can get around this exception?
Regards,
Salman
On Mon, Feb 1, 2016 at 6:08 PM, Susheel Kumar <su...@gmail.com> wrote:
> Post is pretty much similar to GET. You can use any REST Client to try.
> Same select URL & pass below header and put the queries parameters into
> body
>
> POST: http://localhost:8983/solr/techproducts/select
>
> Header
> ==
> Content-Type:application/x-www-form-urlencoded
>
> payload/body:
> ==
> q=*:*&rows=2
>
>
> Thanks,
> Susheel
>
> On Mon, Feb 1, 2016 at 2:38 AM, Salman Ansari <sa...@gmail.com>
> wrote:
>
> > Cool. I would give POST a try. Any samples of using Post while passing
> the
> > query string values (such as ORing between Solr field values) using
> > Solr.NET?
> >
> > Regards,
> > Salman
> >
> > On Sun, Jan 31, 2016 at 10:21 PM, Shawn Heisey <ap...@elyograg.org>
> > wrote:
> >
> > > On 1/31/2016 7:20 AM, Salman Ansari wrote:
> > > > I am building a long query containing multiple ORs between query
> > terms. I
> > > > started to receive the following exception:
> > > >
> > > > The remote server returned an error: (414) Request-URI Too Long. Any
> > idea
> > > > what is the limit of the URL in Solr? Moreover, as a solution I was
> > > > thinking of chunking the query into multiple requests but I was
> > wondering
> > > > if anyone has a better approach?
> > >
> > > The default HTTP header size limit on most webservers and containers
> > > (including the Jetty that ships with Solr) is 8192 bytes. A typical
> > > request like this will start with "GET " and end with " HTTP/1.1",
> which
> > > count against that 8192 bytes. The max header size can be increased.
> > >
> > > If you place the parameters into a POST request instead of on the URL,
> > > then the default size limit of that POST request in Solr is 2MB. This
> > > can also be increased.
> > >
> > > Thanks,
> > > Shawn
> > >
> > >
> >
>
Re: URI is too long
Posted by Susheel Kumar <su...@gmail.com>.
Post is pretty much similar to GET. You can use any REST Client to try.
Same select URL & pass below header and put the queries parameters into body
POST: http://localhost:8983/solr/techproducts/select
Header
==
Content-Type:application/x-www-form-urlencoded
payload/body:
==
q=*:*&rows=2
Thanks,
Susheel
On Mon, Feb 1, 2016 at 2:38 AM, Salman Ansari <sa...@gmail.com>
wrote:
> Cool. I would give POST a try. Any samples of using Post while passing the
> query string values (such as ORing between Solr field values) using
> Solr.NET?
>
> Regards,
> Salman
>
> On Sun, Jan 31, 2016 at 10:21 PM, Shawn Heisey <ap...@elyograg.org>
> wrote:
>
> > On 1/31/2016 7:20 AM, Salman Ansari wrote:
> > > I am building a long query containing multiple ORs between query
> terms. I
> > > started to receive the following exception:
> > >
> > > The remote server returned an error: (414) Request-URI Too Long. Any
> idea
> > > what is the limit of the URL in Solr? Moreover, as a solution I was
> > > thinking of chunking the query into multiple requests but I was
> wondering
> > > if anyone has a better approach?
> >
> > The default HTTP header size limit on most webservers and containers
> > (including the Jetty that ships with Solr) is 8192 bytes. A typical
> > request like this will start with "GET " and end with " HTTP/1.1", which
> > count against that 8192 bytes. The max header size can be increased.
> >
> > If you place the parameters into a POST request instead of on the URL,
> > then the default size limit of that POST request in Solr is 2MB. This
> > can also be increased.
> >
> > Thanks,
> > Shawn
> >
> >
>
Re: URI is too long
Posted by Salman Ansari <sa...@gmail.com>.
Cool. I would give POST a try. Any samples of using Post while passing the
query string values (such as ORing between Solr field values) using
Solr.NET?
Regards,
Salman
On Sun, Jan 31, 2016 at 10:21 PM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 1/31/2016 7:20 AM, Salman Ansari wrote:
> > I am building a long query containing multiple ORs between query terms. I
> > started to receive the following exception:
> >
> > The remote server returned an error: (414) Request-URI Too Long. Any idea
> > what is the limit of the URL in Solr? Moreover, as a solution I was
> > thinking of chunking the query into multiple requests but I was wondering
> > if anyone has a better approach?
>
> The default HTTP header size limit on most webservers and containers
> (including the Jetty that ships with Solr) is 8192 bytes. A typical
> request like this will start with "GET " and end with " HTTP/1.1", which
> count against that 8192 bytes. The max header size can be increased.
>
> If you place the parameters into a POST request instead of on the URL,
> then the default size limit of that POST request in Solr is 2MB. This
> can also be increased.
>
> Thanks,
> Shawn
>
>
Re: URI is too long
Posted by Shawn Heisey <ap...@elyograg.org>.
On 1/31/2016 7:20 AM, Salman Ansari wrote:
> I am building a long query containing multiple ORs between query terms. I
> started to receive the following exception:
>
> The remote server returned an error: (414) Request-URI Too Long. Any idea
> what is the limit of the URL in Solr? Moreover, as a solution I was
> thinking of chunking the query into multiple requests but I was wondering
> if anyone has a better approach?
The default HTTP header size limit on most webservers and containers
(including the Jetty that ships with Solr) is 8192 bytes. A typical
request like this will start with "GET " and end with " HTTP/1.1", which
count against that 8192 bytes. The max header size can be increased.
If you place the parameters into a POST request instead of on the URL,
then the default size limit of that POST request in Solr is 2MB. This
can also be increased.
Thanks,
Shawn
Re: URI is too long
Posted by Jack Krupansky <ja...@gmail.com>.
Or try the terms query parser that lets you eliminate all the OR operators:
https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-TermsQueryParser
-- Jack Krupansky
On Sun, Jan 31, 2016 at 9:23 AM, Paul Libbrecht <pa...@hoplahup.net> wrote:
> How about using POST?
>
> paul
>
> > Salman Ansari <ma...@gmail.com>
> > 31 January 2016 at 15:20
> > Hi,
> >
> > I am building a long query containing multiple ORs between query terms. I
> > started to receive the following exception:
> >
> > The remote server returned an error: (414) Request-URI Too Long. Any idea
> > what is the limit of the URL in Solr? Moreover, as a solution I was
> > thinking of chunking the query into multiple requests but I was wondering
> > if anyone has a better approach?
> >
> > Regards,
> > Salman
> >
>
>
Re: URI is too long
Posted by Paul Libbrecht <pa...@hoplahup.net>.
How about using POST?
paul
> Salman Ansari <ma...@gmail.com>
> 31 January 2016 at 15:20
> Hi,
>
> I am building a long query containing multiple ORs between query terms. I
> started to receive the following exception:
>
> The remote server returned an error: (414) Request-URI Too Long. Any idea
> what is the limit of the URL in Solr? Moreover, as a solution I was
> thinking of chunking the query into multiple requests but I was wondering
> if anyone has a better approach?
>
> Regards,
> Salman
>