You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Tomer Shahar <ts...@varonis.com> on 2019/07/04 12:17:23 UTC

Creating HttpEntityEnclosingRequestBase with a repeatable entity

Hi. I'm using solrj (7.3.1).

I encountered an error for delete queries that fail on an unauthorized exception. I noticed other requests succeed.

I managed to track it down to NTLM authentications. org.apache.http.impl.execchain.MainClientExec (line 315) will remove authentication headers before the retry, so for any of the other requests, the second attempt passes.

Before getting into why it fails on the first attempt, I needed to understand what makes the delete queries special. Those request were never attempted a second time.

It got me to this in HttpSolrClient, in createMethod:


if(contentWriter != null) {
  String fullQueryUrl = url + wparams.toQueryString();
  HttpEntityEnclosingRequestBase postOrPut = SolrRequest.METHOD.POST == request.getMethod() ?
      new HttpPost(fullQueryUrl) : new HttpPut(fullQueryUrl);
  postOrPut.addHeader("Content-Type",
      contentWriter.getContentType());
  postOrPut.setEntity(new BasicHttpEntity(){
    @Override
    public boolean isStreaming() {
      return true;
    }

    @Override
    public void writeTo(OutputStream outstream) throws IOException {
      contentWriter.write(outstream);
    }

  });
  return postOrPut;

I changed the BasicHttpEntity to be repeatable, as follows


if(contentWriter != null) {
  String fullQueryUrl = url + wparams.toQueryString();
  HttpEntityEnclosingRequestBase postOrPut = SolrRequest.METHOD.POST == request.getMethod() ?
      new HttpPost(fullQueryUrl) : new HttpPut(fullQueryUrl);
  postOrPut.addHeader("Content-Type",
      contentWriter.getContentType());
  postOrPut.setEntity(new BasicHttpEntity(){
    @Override
    public boolean isStreaming() {
      return true;
    }

    @Override
    public void writeTo(OutputStream outstream) throws IOException {
      contentWriter.write(outstream);
    }

    @Override
    public boolean isRepeatable() {
      return true;
    }
  });
  return postOrPut;


This changed allowed delete queries to work on the second attempt as the other requests.


My questions:


  1.  Is there a reason for the entity to NOT be repeatable?
  2.  It seems that the authentication headers that are added are implemented in httpclient-4.5.3 which solr depends on. Is there a way for me to configure it from SolrJ?


________________________________
This email and any attachments thereto may contain private, confidential, and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments thereto) by others is strictly prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete the original and any copies of this email and any attachments thereto.

Creating HttpEntityEnclosingRequestBase with a repeatable entity

Posted by Tomer Shahar <ts...@varonis.com>.
Hi. I'm using solrj (7.3.1).

I encountered an error for delete queries that fail on an unauthorized exception. I noticed other requests succeed.

I managed to track it down to NTLM authentications. org.apache.http.impl.execchain.MainClientExec (line 315) will remove authentication headers before the retry, so for any of the other requests, the second attempt passes.

Before getting into why it fails on the first attempt, I needed to understand what makes the delete queries special. Those request were never attempted a second time.

It got me to this in HttpSolrClient, in createMethod:


if(contentWriter != null) {
  String fullQueryUrl = url + wparams.toQueryString();
  HttpEntityEnclosingRequestBase postOrPut = SolrRequest.METHOD.POST == request.getMethod() ?
      new HttpPost(fullQueryUrl) : new HttpPut(fullQueryUrl);
  postOrPut.addHeader("Content-Type",
      contentWriter.getContentType());
  postOrPut.setEntity(new BasicHttpEntity(){
    @Override
    public boolean isStreaming() {
      return true;
    }

    @Override
    public void writeTo(OutputStream outstream) throws IOException {
      contentWriter.write(outstream);
    }

  });
  return postOrPut;

I changed the BasicHttpEntity to be repeatable, as follows


if(contentWriter != null) {
  String fullQueryUrl = url + wparams.toQueryString();
  HttpEntityEnclosingRequestBase postOrPut = SolrRequest.METHOD.POST == request.getMethod() ?
      new HttpPost(fullQueryUrl) : new HttpPut(fullQueryUrl);
  postOrPut.addHeader("Content-Type",
      contentWriter.getContentType());
  postOrPut.setEntity(new BasicHttpEntity(){
    @Override
    public boolean isStreaming() {
      return true;
    }

    @Override
    public void writeTo(OutputStream outstream) throws IOException {
      contentWriter.write(outstream);
    }

    @Override
    public boolean isRepeatable() {
      return true;
    }
  });
  return postOrPut;


This changed allowed delete queries to work on the second attempt as the other requests.


My questions:


  1.  Is there a reason for the entity to NOT be repeatable?
  2.  It seems that the authentication headers that are added are implemented in httpclient-4.5.3 which solr depends on. Is there a way for me to configure it from SolrJ?


________________________________
This email and any attachments thereto may contain private, confidential, and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments thereto) by others is strictly prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete the original and any copies of this email and any attachments thereto.