You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by Karl Wright <da...@gmail.com> on 2018/09/21 14:15:03 UTC
How are you supposed to send multipart form requests using SolrJ now?
Hi all,
With the release of 7.4, several things changed inside Solrj. One critical
thing that changed was that it now seems to be impossible to send POST
requests with multipart form encoding. This functionality is critical to
ManifoldCF since the metadata it sends to Solr regularly exceeds URL
length. Here's a snippet of (modified) code from HttpSolrClient:
>>>>>>
if(contentWriter != null) {
String fullQueryUrl = url + toQueryString(wparams, false);
HttpEntityEnclosingRequestBase postOrPut = SolrRequest.METHOD.POST
== request.getMethod() ?
new HttpPost(fullQueryUrl) : new HttpPut(fullQueryUrl);
System.out.println("Content type is
"+contentWriter.getContentType());
postOrPut.addHeader("Content-Type",
contentWriter.getContentType());
postOrPut.setEntity(new BasicHttpEntity(){
@Override
public boolean isStreaming() {
return true;
}
@Override
public void writeTo(OutputStream outstream) throws IOException {
contentWriter.write(outstream);
}
});
return postOrPut;
<<<<<<
This takes over all transmissions of POST requests, in practice, and
transmits them as application/octet-stream, with metadata on the URL.
There's no way to override this; while I can instead used the multipart
stuff consistently, when I do so the request to delete a document fails,
with this SolrJ stack trace:
>>>>>>
FATAL 2018-09-21T09:54:58,362 (Document delete thread '4') - Error tossed:
This Should not happen
java.lang.RuntimeException: This Should not happen
at
org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStreams(BinaryRequestWriter.java:67)
~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
jpountz - 2018-06-18 16:55:14]
at
org.apache.manifoldcf.agents.output.solr.ModifiedHttpSolrClient.createMethod(ModifiedHttpSolrClient.java:148)
~[classes/:?]
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
jpountz - 2018-06-18 16:55:14]
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
jpountz - 2018-06-18 16:55:14]
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
jpountz - 2018-06-18 16:55:14]
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
jpountz - 2018-06-18 16:55:14]
at
org.apache.manifoldcf.agents.output.solr.HttpPoster$DeleteThread.run(HttpPoster.java:1364)
~[classes/:?]
<<<<<<
Any ideas?
Karl
Re: How are you supposed to send multipart form requests using SolrJ
now?
Posted by Shawn Heisey <ap...@elyograg.org>.
On 9/24/2018 2:53 AM, Karl Wright wrote:
> I also tried to tag Paul Nobel in the ticket but Jira does not
> recognize that name.
The name you're after is "Noble Paul" ... not "Paul Nobel".
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: How are you supposed to send multipart form requests using SolrJ now?
Posted by Karl Wright <da...@gmail.com>.
I've created a ticket for this:
SOLR-12798
It seems to me that not only is the loss of multipart form posting a
serious matter, but also the fact that you cannot use a multipart form post
for any UpdateRequest. Both are described in the ticket.
This is a major problem for the ManifoldCF release cycle happening now. If
there is something I've missed and there is a good way to accomplish this
in SolrJ, please let me know.
I also tried to tag Paul Nobel in the ticket but Jira does not recognize
that name.
Karl
On Mon, Sep 24, 2018 at 4:01 AM Karl Wright <da...@gmail.com> wrote:
> Ok, the commit that broke this was made on 10-31-2107 by Paul Nobel:
>
> >>>>>>
> f6f6f113209 (Noble Paul 2017-06-16 14:05:51 +0930 327)
> protected HttpRequestBase createMethod(SolrRequest request, String
> collection) throws IOException, SolrServerException {
> f6f6f113209 (Noble Paul 2017-06-16 14:05:51 +0930 328) if
> (request instanceof V2RequestSupport) {
> f6f6f113209 (Noble Paul 2017-06-16 14:05:51 +0930 329)
> request = ((V2RequestSupport) request).getV2Request();
> f6f6f113209 (Noble Paul 2017-06-16 14:05:51 +0930 330) }
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 331)
> SolrParams params = request.getParams();
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 332)
> RequestWriter.ContentWriter contentWriter =
> requestWriter.getContentWriter(request);
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 333)
> Collection<ContentStream> streams = contentWriter == null ?
> requestWriter.getContentStreams(request) : null;
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 334)
> String path = requestWriter.getPath(request);
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 335) if
> (path == null || !path.startsWith("/")) {
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 336)
> path = DEFAULT_PATH;
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 337) }
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 338)
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 339)
> ResponseParser parser = request.getResponseParser();
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 340) if
> (parser == null) {
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 341)
> parser = this.parser;
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 342) }
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 343)
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 344) //
> The parser 'wt=' and 'version=' params are used instead of the original
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 345) //
> params
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 346)
> ModifiableSolrParams wparams = new ModifiableSolrParams(params);
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 347) if
> (parser != null) {
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 348)
> wparams.set(CommonParams.WT, parser.getWriterType());
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 349)
> wparams.set(CommonParams.VERSION, parser.getVersion());
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 350) }
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 351) if
> (invariantParams != null) {
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 352)
> wparams.add(invariantParams);
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 353) }
> 68d180346ab (Alan Woodward 2015-03-09 11:54:45 +0000 354)
> 68d180346ab (Alan Woodward 2015-03-09 11:54:45 +0000 355)
> String basePath = baseUrl;
> 68d180346ab (Alan Woodward 2015-03-09 11:54:45 +0000 356) if
> (collection != null)
> 68d180346ab (Alan Woodward 2015-03-09 11:54:45 +0000 357)
> basePath += "/" + collection;
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 358)
> cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 359) if
> (request instanceof V2Request) {
> cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 360) if
> (System.getProperty("solr.v2RealPath") == null) {
> c8e0e939e49 (Ishan Chattopadhyaya 2017-08-20 21:00:15 +0530 361)
> basePath = baseUrl.replace("/solr", "/api");
> cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 362) }
> else {
> cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 363)
> basePath = baseUrl + "/____v2";
> cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 364) }
> cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 365) }
> cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 366)
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 367) if
> (SolrRequest.METHOD.GET == request.getMethod()) {
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 368) if
> (streams != null || contentWriter != null) {
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 369)
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "GET can't
> send streams!");
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 370) }
> ce172acb8fe (markrmiller 2016-04-01 12:21:59 -0400 371)
> f4fe17cce85 (Uwe Schindler 2015-11-22 09:56:22 +0000 372)
> return new HttpGet(basePath + path + wparams.toQueryString());
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 373) }
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 374)
> cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 375) if
> (SolrRequest.METHOD.DELETE == request.getMethod()) {
> cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 376)
> return new HttpDelete(basePath + path + wparams.toQueryString());
> cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 377) }
> cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 378)
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 379) if
> (SolrRequest.METHOD.POST == request.getMethod() || SolrRequest.METHOD.PUT
> == request.getMethod()) {
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 380)
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 381)
> String url = basePath + path;
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 382)
> boolean hasNullStreamName = false;
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 383) if
> (streams != null) {
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 384)
> for (ContentStream cs : streams) {
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 385)
> if (cs.getName() == null) {
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 386)
> hasNullStreamName = true;
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 387)
> break;
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 388)
> }
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 389) }
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 390) }
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 391)
> boolean isMultipart = ((this.useMultiPartPost && SolrRequest.METHOD.POST
> == request.getMethod())
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 392)
> || (streams != null && streams.size() > 1)) && !hasNullStreamName;
> ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 393)
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 394)
> LinkedList<NameValuePair> postOrPutParams = new LinkedList<>();
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 395)
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 396)
> if(contentWriter != null) {
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 397)
> String fullQueryUrl = url + wparams.toQueryString();
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 398)
> HttpEntityEnclosingRequestBase postOrPut = SolrRequest.METHOD.POST ==
> request.getMethod() ?
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 399)
> new HttpPost(fullQueryUrl) : new HttpPut(fullQueryUrl);
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 400)
> postOrPut.addHeader("Content-Type",
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 401)
> contentWriter.getContentType());
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 402)
> postOrPut.setEntity(new BasicHttpEntity(){
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 403)
> @Override
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 404)
> public boolean isStreaming() {
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 405)
> return true;
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 406)
> }
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 407)
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 408)
> @Override
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 409)
> public void writeTo(OutputStream outstream) throws IOException {
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 410)
> contentWriter.write(outstream);
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 411)
> }
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 412)
> });
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 413)
> return postOrPut;
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 414)
> 706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 415) }
> else if (streams == null || isMultipart) {
> 203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 416)
> // send server list and request list as query string params
> ...
> <<<<<<
>
> The introduction of RequestWriter essentially prevented use of streams and
> multipart EXCEPT when there was no RequestWriter in a request, which is
> never.
>
> Karl
>
>
> On Sat, Sep 22, 2018 at 12:19 PM Karl Wright <da...@gmail.com> wrote:
>
>> I'll dig to find which commit it was that introduced the changes.
>> Karl
>>
>> On Sat, Sep 22, 2018 at 11:28 AM Alexandre Rafalovitch <
>> arafalov@gmail.com> wrote:
>>
>>> Do you know which Jira introduced the change?
>>>
>>> Without further details (I am barely familiar with that code part), it
>>> sounds like a regression and possibly even a blocker for the next
>>> version. Unless there is a workaround that is quite non-intuitive.
>>>
>>> Regards,
>>> Alex.
>>>
>>> On 21 September 2018 at 11:43, Karl Wright <da...@gmail.com> wrote:
>>> > I should also mention that Solr Cell makes use of the multipart form's
>>> > separator name. This functionality seems to be unavailable through
>>> the URL.
>>> > We have a number of clients who rely on this.
>>> >
>>> > Karl
>>> >
>>> >
>>> > On Fri, Sep 21, 2018 at 10:15 AM Karl Wright <da...@gmail.com>
>>> wrote:
>>> >>
>>> >> Hi all,
>>> >>
>>> >> With the release of 7.4, several things changed inside Solrj. One
>>> >> critical thing that changed was that it now seems to be impossible to
>>> send
>>> >> POST requests with multipart form encoding. This functionality is
>>> critical
>>> >> to ManifoldCF since the metadata it sends to Solr regularly exceeds
>>> URL
>>> >> length. Here's a snippet of (modified) code from HttpSolrClient:
>>> >>
>>> >> >>>>>>
>>> >> if(contentWriter != null) {
>>> >> String fullQueryUrl = url + toQueryString(wparams, false);
>>> >> HttpEntityEnclosingRequestBase postOrPut =
>>> SolrRequest.METHOD.POST
>>> >> == request.getMethod() ?
>>> >> new HttpPost(fullQueryUrl) : new HttpPut(fullQueryUrl);
>>> >> System.out.println("Content type is
>>> >> "+contentWriter.getContentType());
>>> >> postOrPut.addHeader("Content-Type",
>>> >> contentWriter.getContentType());
>>> >> postOrPut.setEntity(new BasicHttpEntity(){
>>> >> @Override
>>> >> public boolean isStreaming() {
>>> >> return true;
>>> >> }
>>> >>
>>> >> @Override
>>> >> public void writeTo(OutputStream outstream) throws
>>> IOException {
>>> >> contentWriter.write(outstream);
>>> >> }
>>> >> });
>>> >> return postOrPut;
>>> >> <<<<<<
>>> >>
>>> >> This takes over all transmissions of POST requests, in practice, and
>>> >> transmits them as application/octet-stream, with metadata on the URL.
>>> >> There's no way to override this; while I can instead used the
>>> multipart
>>> >> stuff consistently, when I do so the request to delete a document
>>> fails,
>>> >> with this SolrJ stack trace:
>>> >>
>>> >> >>>>>>
>>> >> FATAL 2018-09-21T09:54:58,362 (Document delete thread '4') - Error
>>> tossed:
>>> >> This Should not happen
>>> >> java.lang.RuntimeException: This Should not happen
>>> >> at
>>> >>
>>> org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStreams(BinaryRequestWriter.java:67)
>>> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc
>>> -
>>> >> jpountz - 2018-06-18 16:55:14]
>>> >> at
>>> >>
>>> org.apache.manifoldcf.agents.output.solr.ModifiedHttpSolrClient.createMethod(ModifiedHttpSolrClient.java:148)
>>> >> ~[classes/:?]
>>> >> at
>>> >>
>>> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
>>> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc
>>> -
>>> >> jpountz - 2018-06-18 16:55:14]
>>> >> at
>>> >>
>>> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
>>> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc
>>> -
>>> >> jpountz - 2018-06-18 16:55:14]
>>> >> at
>>> >> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
>>> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc
>>> -
>>> >> jpountz - 2018-06-18 16:55:14]
>>> >> at
>>> >> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
>>> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc
>>> -
>>> >> jpountz - 2018-06-18 16:55:14]
>>> >> at
>>> >>
>>> org.apache.manifoldcf.agents.output.solr.HttpPoster$DeleteThread.run(HttpPoster.java:1364)
>>> >> ~[classes/:?]
>>> >> <<<<<<
>>> >>
>>> >> Any ideas?
>>> >>
>>> >> Karl
>>> >>
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
Re: How are you supposed to send multipart form requests using SolrJ now?
Posted by Karl Wright <da...@gmail.com>.
Ok, the commit that broke this was made on 10-31-2107 by Paul Nobel:
>>>>>>
f6f6f113209 (Noble Paul 2017-06-16 14:05:51 +0930 327)
protected HttpRequestBase createMethod(SolrRequest request, String
collection) throws IOException, SolrServerException {
f6f6f113209 (Noble Paul 2017-06-16 14:05:51 +0930 328) if
(request instanceof V2RequestSupport) {
f6f6f113209 (Noble Paul 2017-06-16 14:05:51 +0930 329)
request = ((V2RequestSupport) request).getV2Request();
f6f6f113209 (Noble Paul 2017-06-16 14:05:51 +0930 330) }
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 331)
SolrParams params = request.getParams();
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 332)
RequestWriter.ContentWriter contentWriter =
requestWriter.getContentWriter(request);
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 333)
Collection<ContentStream> streams = contentWriter == null ?
requestWriter.getContentStreams(request) : null;
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 334)
String path = requestWriter.getPath(request);
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 335) if
(path == null || !path.startsWith("/")) {
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 336)
path = DEFAULT_PATH;
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 337) }
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 338)
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 339)
ResponseParser parser = request.getResponseParser();
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 340) if
(parser == null) {
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 341)
parser = this.parser;
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 342) }
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 343)
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 344) //
The parser 'wt=' and 'version=' params are used instead of the original
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 345) //
params
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 346)
ModifiableSolrParams wparams = new ModifiableSolrParams(params);
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 347) if
(parser != null) {
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 348)
wparams.set(CommonParams.WT, parser.getWriterType());
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 349)
wparams.set(CommonParams.VERSION, parser.getVersion());
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 350) }
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 351) if
(invariantParams != null) {
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 352)
wparams.add(invariantParams);
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 353) }
68d180346ab (Alan Woodward 2015-03-09 11:54:45 +0000 354)
68d180346ab (Alan Woodward 2015-03-09 11:54:45 +0000 355)
String basePath = baseUrl;
68d180346ab (Alan Woodward 2015-03-09 11:54:45 +0000 356) if
(collection != null)
68d180346ab (Alan Woodward 2015-03-09 11:54:45 +0000 357)
basePath += "/" + collection;
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 358)
cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 359) if
(request instanceof V2Request) {
cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 360) if
(System.getProperty("solr.v2RealPath") == null) {
c8e0e939e49 (Ishan Chattopadhyaya 2017-08-20 21:00:15 +0530 361)
basePath = baseUrl.replace("/solr", "/api");
cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 362) }
else {
cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 363)
basePath = baseUrl + "/____v2";
cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 364) }
cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 365) }
cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 366)
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 367) if
(SolrRequest.METHOD.GET == request.getMethod()) {
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 368) if
(streams != null || contentWriter != null) {
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 369)
throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "GET can't
send streams!");
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 370) }
ce172acb8fe (markrmiller 2016-04-01 12:21:59 -0400 371)
f4fe17cce85 (Uwe Schindler 2015-11-22 09:56:22 +0000 372)
return new HttpGet(basePath + path + wparams.toQueryString());
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 373) }
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 374)
cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 375) if
(SolrRequest.METHOD.DELETE == request.getMethod()) {
cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 376)
return new HttpDelete(basePath + path + wparams.toQueryString());
cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 377) }
cc8b5bab0bd (Cao Manh Dat 2017-05-11 09:06:03 +0700 378)
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 379) if
(SolrRequest.METHOD.POST == request.getMethod() || SolrRequest.METHOD.PUT
== request.getMethod()) {
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 380)
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 381)
String url = basePath + path;
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 382)
boolean hasNullStreamName = false;
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 383) if
(streams != null) {
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 384)
for (ContentStream cs : streams) {
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 385)
if (cs.getName() == null) {
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 386)
hasNullStreamName = true;
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 387)
break;
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 388)
}
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 389) }
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 390) }
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 391)
boolean isMultipart = ((this.useMultiPartPost && SolrRequest.METHOD.POST
== request.getMethod())
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 392)
|| (streams != null && streams.size() > 1)) && !hasNullStreamName;
ef809a0f10e (Alan Woodward 2014-12-31 14:05:48 +0000 393)
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 394)
LinkedList<NameValuePair> postOrPutParams = new LinkedList<>();
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 395)
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 396)
if(contentWriter != null) {
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 397)
String fullQueryUrl = url + wparams.toQueryString();
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 398)
HttpEntityEnclosingRequestBase postOrPut = SolrRequest.METHOD.POST ==
request.getMethod() ?
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 399)
new HttpPost(fullQueryUrl) : new HttpPut(fullQueryUrl);
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 400)
postOrPut.addHeader("Content-Type",
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 401)
contentWriter.getContentType());
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 402)
postOrPut.setEntity(new BasicHttpEntity(){
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 403)
@Override
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 404)
public boolean isStreaming() {
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 405)
return true;
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 406)
}
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 407)
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 408)
@Override
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 409)
public void writeTo(OutputStream outstream) throws IOException {
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 410)
contentWriter.write(outstream);
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 411)
}
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 412)
});
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 413)
return postOrPut;
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 414)
706b6c91718 (Noble Paul 2017-10-31 13:16:31 +1030 415) }
else if (streams == null || isMultipart) {
203bbf504a3 (Alan Woodward 2015-03-26 10:42:22 +0000 416)
// send server list and request list as query string params
...
<<<<<<
The introduction of RequestWriter essentially prevented use of streams and
multipart EXCEPT when there was no RequestWriter in a request, which is
never.
Karl
On Sat, Sep 22, 2018 at 12:19 PM Karl Wright <da...@gmail.com> wrote:
> I'll dig to find which commit it was that introduced the changes.
> Karl
>
> On Sat, Sep 22, 2018 at 11:28 AM Alexandre Rafalovitch <ar...@gmail.com>
> wrote:
>
>> Do you know which Jira introduced the change?
>>
>> Without further details (I am barely familiar with that code part), it
>> sounds like a regression and possibly even a blocker for the next
>> version. Unless there is a workaround that is quite non-intuitive.
>>
>> Regards,
>> Alex.
>>
>> On 21 September 2018 at 11:43, Karl Wright <da...@gmail.com> wrote:
>> > I should also mention that Solr Cell makes use of the multipart form's
>> > separator name. This functionality seems to be unavailable through the
>> URL.
>> > We have a number of clients who rely on this.
>> >
>> > Karl
>> >
>> >
>> > On Fri, Sep 21, 2018 at 10:15 AM Karl Wright <da...@gmail.com>
>> wrote:
>> >>
>> >> Hi all,
>> >>
>> >> With the release of 7.4, several things changed inside Solrj. One
>> >> critical thing that changed was that it now seems to be impossible to
>> send
>> >> POST requests with multipart form encoding. This functionality is
>> critical
>> >> to ManifoldCF since the metadata it sends to Solr regularly exceeds URL
>> >> length. Here's a snippet of (modified) code from HttpSolrClient:
>> >>
>> >> >>>>>>
>> >> if(contentWriter != null) {
>> >> String fullQueryUrl = url + toQueryString(wparams, false);
>> >> HttpEntityEnclosingRequestBase postOrPut =
>> SolrRequest.METHOD.POST
>> >> == request.getMethod() ?
>> >> new HttpPost(fullQueryUrl) : new HttpPut(fullQueryUrl);
>> >> System.out.println("Content type is
>> >> "+contentWriter.getContentType());
>> >> postOrPut.addHeader("Content-Type",
>> >> contentWriter.getContentType());
>> >> postOrPut.setEntity(new BasicHttpEntity(){
>> >> @Override
>> >> public boolean isStreaming() {
>> >> return true;
>> >> }
>> >>
>> >> @Override
>> >> public void writeTo(OutputStream outstream) throws
>> IOException {
>> >> contentWriter.write(outstream);
>> >> }
>> >> });
>> >> return postOrPut;
>> >> <<<<<<
>> >>
>> >> This takes over all transmissions of POST requests, in practice, and
>> >> transmits them as application/octet-stream, with metadata on the URL.
>> >> There's no way to override this; while I can instead used the multipart
>> >> stuff consistently, when I do so the request to delete a document
>> fails,
>> >> with this SolrJ stack trace:
>> >>
>> >> >>>>>>
>> >> FATAL 2018-09-21T09:54:58,362 (Document delete thread '4') - Error
>> tossed:
>> >> This Should not happen
>> >> java.lang.RuntimeException: This Should not happen
>> >> at
>> >>
>> org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStreams(BinaryRequestWriter.java:67)
>> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
>> >> jpountz - 2018-06-18 16:55:14]
>> >> at
>> >>
>> org.apache.manifoldcf.agents.output.solr.ModifiedHttpSolrClient.createMethod(ModifiedHttpSolrClient.java:148)
>> >> ~[classes/:?]
>> >> at
>> >>
>> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
>> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
>> >> jpountz - 2018-06-18 16:55:14]
>> >> at
>> >>
>> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
>> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
>> >> jpountz - 2018-06-18 16:55:14]
>> >> at
>> >> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
>> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
>> >> jpountz - 2018-06-18 16:55:14]
>> >> at
>> >> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
>> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
>> >> jpountz - 2018-06-18 16:55:14]
>> >> at
>> >>
>> org.apache.manifoldcf.agents.output.solr.HttpPoster$DeleteThread.run(HttpPoster.java:1364)
>> >> ~[classes/:?]
>> >> <<<<<<
>> >>
>> >> Any ideas?
>> >>
>> >> Karl
>> >>
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
Re: How are you supposed to send multipart form requests using SolrJ now?
Posted by Karl Wright <da...@gmail.com>.
I'll dig to find which commit it was that introduced the changes.
Karl
On Sat, Sep 22, 2018 at 11:28 AM Alexandre Rafalovitch <ar...@gmail.com>
wrote:
> Do you know which Jira introduced the change?
>
> Without further details (I am barely familiar with that code part), it
> sounds like a regression and possibly even a blocker for the next
> version. Unless there is a workaround that is quite non-intuitive.
>
> Regards,
> Alex.
>
> On 21 September 2018 at 11:43, Karl Wright <da...@gmail.com> wrote:
> > I should also mention that Solr Cell makes use of the multipart form's
> > separator name. This functionality seems to be unavailable through the
> URL.
> > We have a number of clients who rely on this.
> >
> > Karl
> >
> >
> > On Fri, Sep 21, 2018 at 10:15 AM Karl Wright <da...@gmail.com> wrote:
> >>
> >> Hi all,
> >>
> >> With the release of 7.4, several things changed inside Solrj. One
> >> critical thing that changed was that it now seems to be impossible to
> send
> >> POST requests with multipart form encoding. This functionality is
> critical
> >> to ManifoldCF since the metadata it sends to Solr regularly exceeds URL
> >> length. Here's a snippet of (modified) code from HttpSolrClient:
> >>
> >> >>>>>>
> >> if(contentWriter != null) {
> >> String fullQueryUrl = url + toQueryString(wparams, false);
> >> HttpEntityEnclosingRequestBase postOrPut =
> SolrRequest.METHOD.POST
> >> == request.getMethod() ?
> >> new HttpPost(fullQueryUrl) : new HttpPut(fullQueryUrl);
> >> System.out.println("Content type is
> >> "+contentWriter.getContentType());
> >> postOrPut.addHeader("Content-Type",
> >> contentWriter.getContentType());
> >> postOrPut.setEntity(new BasicHttpEntity(){
> >> @Override
> >> public boolean isStreaming() {
> >> return true;
> >> }
> >>
> >> @Override
> >> public void writeTo(OutputStream outstream) throws
> IOException {
> >> contentWriter.write(outstream);
> >> }
> >> });
> >> return postOrPut;
> >> <<<<<<
> >>
> >> This takes over all transmissions of POST requests, in practice, and
> >> transmits them as application/octet-stream, with metadata on the URL.
> >> There's no way to override this; while I can instead used the multipart
> >> stuff consistently, when I do so the request to delete a document fails,
> >> with this SolrJ stack trace:
> >>
> >> >>>>>>
> >> FATAL 2018-09-21T09:54:58,362 (Document delete thread '4') - Error
> tossed:
> >> This Should not happen
> >> java.lang.RuntimeException: This Should not happen
> >> at
> >>
> org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStreams(BinaryRequestWriter.java:67)
> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> >> jpountz - 2018-06-18 16:55:14]
> >> at
> >>
> org.apache.manifoldcf.agents.output.solr.ModifiedHttpSolrClient.createMethod(ModifiedHttpSolrClient.java:148)
> >> ~[classes/:?]
> >> at
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> >> jpountz - 2018-06-18 16:55:14]
> >> at
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> >> jpountz - 2018-06-18 16:55:14]
> >> at
> >> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> >> jpountz - 2018-06-18 16:55:14]
> >> at
> >> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
> >> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> >> jpountz - 2018-06-18 16:55:14]
> >> at
> >>
> org.apache.manifoldcf.agents.output.solr.HttpPoster$DeleteThread.run(HttpPoster.java:1364)
> >> ~[classes/:?]
> >> <<<<<<
> >>
> >> Any ideas?
> >>
> >> Karl
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: How are you supposed to send multipart form requests using SolrJ now?
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
Do you know which Jira introduced the change?
Without further details (I am barely familiar with that code part), it
sounds like a regression and possibly even a blocker for the next
version. Unless there is a workaround that is quite non-intuitive.
Regards,
Alex.
On 21 September 2018 at 11:43, Karl Wright <da...@gmail.com> wrote:
> I should also mention that Solr Cell makes use of the multipart form's
> separator name. This functionality seems to be unavailable through the URL.
> We have a number of clients who rely on this.
>
> Karl
>
>
> On Fri, Sep 21, 2018 at 10:15 AM Karl Wright <da...@gmail.com> wrote:
>>
>> Hi all,
>>
>> With the release of 7.4, several things changed inside Solrj. One
>> critical thing that changed was that it now seems to be impossible to send
>> POST requests with multipart form encoding. This functionality is critical
>> to ManifoldCF since the metadata it sends to Solr regularly exceeds URL
>> length. Here's a snippet of (modified) code from HttpSolrClient:
>>
>> >>>>>>
>> if(contentWriter != null) {
>> String fullQueryUrl = url + toQueryString(wparams, false);
>> HttpEntityEnclosingRequestBase postOrPut = SolrRequest.METHOD.POST
>> == request.getMethod() ?
>> new HttpPost(fullQueryUrl) : new HttpPut(fullQueryUrl);
>> System.out.println("Content type is
>> "+contentWriter.getContentType());
>> postOrPut.addHeader("Content-Type",
>> contentWriter.getContentType());
>> postOrPut.setEntity(new BasicHttpEntity(){
>> @Override
>> public boolean isStreaming() {
>> return true;
>> }
>>
>> @Override
>> public void writeTo(OutputStream outstream) throws IOException {
>> contentWriter.write(outstream);
>> }
>> });
>> return postOrPut;
>> <<<<<<
>>
>> This takes over all transmissions of POST requests, in practice, and
>> transmits them as application/octet-stream, with metadata on the URL.
>> There's no way to override this; while I can instead used the multipart
>> stuff consistently, when I do so the request to delete a document fails,
>> with this SolrJ stack trace:
>>
>> >>>>>>
>> FATAL 2018-09-21T09:54:58,362 (Document delete thread '4') - Error tossed:
>> This Should not happen
>> java.lang.RuntimeException: This Should not happen
>> at
>> org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStreams(BinaryRequestWriter.java:67)
>> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
>> jpountz - 2018-06-18 16:55:14]
>> at
>> org.apache.manifoldcf.agents.output.solr.ModifiedHttpSolrClient.createMethod(ModifiedHttpSolrClient.java:148)
>> ~[classes/:?]
>> at
>> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
>> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
>> jpountz - 2018-06-18 16:55:14]
>> at
>> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
>> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
>> jpountz - 2018-06-18 16:55:14]
>> at
>> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
>> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
>> jpountz - 2018-06-18 16:55:14]
>> at
>> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
>> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
>> jpountz - 2018-06-18 16:55:14]
>> at
>> org.apache.manifoldcf.agents.output.solr.HttpPoster$DeleteThread.run(HttpPoster.java:1364)
>> ~[classes/:?]
>> <<<<<<
>>
>> Any ideas?
>>
>> Karl
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: How are you supposed to send multipart form requests using SolrJ now?
Posted by Karl Wright <da...@gmail.com>.
I should also mention that Solr Cell makes use of the multipart form's
separator name. This functionality seems to be unavailable through the
URL. We have a number of clients who rely on this.
Karl
On Fri, Sep 21, 2018 at 10:15 AM Karl Wright <da...@gmail.com> wrote:
> Hi all,
>
> With the release of 7.4, several things changed inside Solrj. One
> critical thing that changed was that it now seems to be impossible to send
> POST requests with multipart form encoding. This functionality is critical
> to ManifoldCF since the metadata it sends to Solr regularly exceeds URL
> length. Here's a snippet of (modified) code from HttpSolrClient:
>
> >>>>>>
> if(contentWriter != null) {
> String fullQueryUrl = url + toQueryString(wparams, false);
> HttpEntityEnclosingRequestBase postOrPut = SolrRequest.METHOD.POST
> == request.getMethod() ?
> new HttpPost(fullQueryUrl) : new HttpPut(fullQueryUrl);
> System.out.println("Content type is
> "+contentWriter.getContentType());
> postOrPut.addHeader("Content-Type",
> contentWriter.getContentType());
> postOrPut.setEntity(new BasicHttpEntity(){
> @Override
> public boolean isStreaming() {
> return true;
> }
>
> @Override
> public void writeTo(OutputStream outstream) throws IOException {
> contentWriter.write(outstream);
> }
> });
> return postOrPut;
> <<<<<<
>
> This takes over all transmissions of POST requests, in practice, and
> transmits them as application/octet-stream, with metadata on the URL.
> There's no way to override this; while I can instead used the multipart
> stuff consistently, when I do so the request to delete a document fails,
> with this SolrJ stack trace:
>
> >>>>>>
> FATAL 2018-09-21T09:54:58,362 (Document delete thread '4') - Error tossed:
> This Should not happen
> java.lang.RuntimeException: This Should not happen
> at
> org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStreams(BinaryRequestWriter.java:67)
> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> jpountz - 2018-06-18 16:55:14]
> at
> org.apache.manifoldcf.agents.output.solr.ModifiedHttpSolrClient.createMethod(ModifiedHttpSolrClient.java:148)
> ~[classes/:?]
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> jpountz - 2018-06-18 16:55:14]
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> jpountz - 2018-06-18 16:55:14]
> at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> jpountz - 2018-06-18 16:55:14]
> at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> jpountz - 2018-06-18 16:55:14]
> at
> org.apache.manifoldcf.agents.output.solr.HttpPoster$DeleteThread.run(HttpPoster.java:1364)
> ~[classes/:?]
> <<<<<<
>
> Any ideas?
>
> Karl
>
>
Re: How are you supposed to send multipart form requests using SolrJ now?
Posted by Karl Wright <da...@gmail.com>.
I should also mention that Solr Cell makes use of the multipart form's
separator name. This functionality seems to be unavailable through the
URL. We have a number of clients who rely on this.
Karl
On Fri, Sep 21, 2018 at 10:15 AM Karl Wright <da...@gmail.com> wrote:
> Hi all,
>
> With the release of 7.4, several things changed inside Solrj. One
> critical thing that changed was that it now seems to be impossible to send
> POST requests with multipart form encoding. This functionality is critical
> to ManifoldCF since the metadata it sends to Solr regularly exceeds URL
> length. Here's a snippet of (modified) code from HttpSolrClient:
>
> >>>>>>
> if(contentWriter != null) {
> String fullQueryUrl = url + toQueryString(wparams, false);
> HttpEntityEnclosingRequestBase postOrPut = SolrRequest.METHOD.POST
> == request.getMethod() ?
> new HttpPost(fullQueryUrl) : new HttpPut(fullQueryUrl);
> System.out.println("Content type is
> "+contentWriter.getContentType());
> postOrPut.addHeader("Content-Type",
> contentWriter.getContentType());
> postOrPut.setEntity(new BasicHttpEntity(){
> @Override
> public boolean isStreaming() {
> return true;
> }
>
> @Override
> public void writeTo(OutputStream outstream) throws IOException {
> contentWriter.write(outstream);
> }
> });
> return postOrPut;
> <<<<<<
>
> This takes over all transmissions of POST requests, in practice, and
> transmits them as application/octet-stream, with metadata on the URL.
> There's no way to override this; while I can instead used the multipart
> stuff consistently, when I do so the request to delete a document fails,
> with this SolrJ stack trace:
>
> >>>>>>
> FATAL 2018-09-21T09:54:58,362 (Document delete thread '4') - Error tossed:
> This Should not happen
> java.lang.RuntimeException: This Should not happen
> at
> org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStreams(BinaryRequestWriter.java:67)
> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> jpountz - 2018-06-18 16:55:14]
> at
> org.apache.manifoldcf.agents.output.solr.ModifiedHttpSolrClient.createMethod(ModifiedHttpSolrClient.java:148)
> ~[classes/:?]
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> jpountz - 2018-06-18 16:55:14]
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> jpountz - 2018-06-18 16:55:14]
> at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> jpountz - 2018-06-18 16:55:14]
> at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
> ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
> jpountz - 2018-06-18 16:55:14]
> at
> org.apache.manifoldcf.agents.output.solr.HttpPoster$DeleteThread.run(HttpPoster.java:1364)
> ~[classes/:?]
> <<<<<<
>
> Any ideas?
>
> Karl
>
>