You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Sean Timm (JIRA)" <ji...@apache.org> on 2008/01/18 22:42:33 UTC
[jira] Created: (SOLR-462) Performance related enhancements to
Solrj
Performance related enhancements to Solrj
-----------------------------------------
Key: SOLR-462
URL: https://issues.apache.org/jira/browse/SOLR-462
Project: Solr
Issue Type: New Feature
Components: clients - java
Reporter: Sean Timm
Changes to CommonsHttpSolrServer.java to add soTimeout (read timeout), connection pool timeout, directive to not follow HTTP redirects, configurable retries on NoHttpResponseException, compression, and not creating a new HttpClient on each request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-462) Performance related enhancements to
Solrj
Posted by "Sean Timm (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Timm updated SOLR-462:
---------------------------
Attachment: solrj-SOLR-462.patch
patch applies to client/java/solrj/src/org/apache/solr/client/solrj/impl/CommonsHttpSolrServer.java
> Performance related enhancements to Solrj
> -----------------------------------------
>
> Key: SOLR-462
> URL: https://issues.apache.org/jira/browse/SOLR-462
> Project: Solr
> Issue Type: New Feature
> Components: clients - java
> Reporter: Sean Timm
> Attachments: solrj-SOLR-462.patch
>
>
> Changes to CommonsHttpSolrServer.java to add soTimeout (read timeout), connection pool timeout, directive to not follow HTTP redirects, configurable retries on NoHttpResponseException, compression, and not creating a new HttpClient on each request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-462) Performance related enhancements to
Solrj
Posted by "Sean Timm (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561189#action_12561189 ]
Sean Timm commented on SOLR-462:
--------------------------------
It's not a big deal to me either way. It just seems that if you can save the bytes, you might as well. In our case for one of our sites, we get ~25MM queries per day. We don't use compression. Sending the request header would result in an extra 725 MB of data (29 bytes * 25MM) over the course of a day (which would be ignored). Compared to the GBs of actual data, it's not much, but a penny saved is a penny earned. :-)
> Performance related enhancements to Solrj
> -----------------------------------------
>
> Key: SOLR-462
> URL: https://issues.apache.org/jira/browse/SOLR-462
> Project: Solr
> Issue Type: New Feature
> Components: clients - java
> Reporter: Sean Timm
> Assignee: Ryan McKinley
> Fix For: 1.3
>
> Attachments: solrj-SOLR-462.patch
>
>
> Changes to CommonsHttpSolrServer.java to add soTimeout (read timeout), connection pool timeout, directive to not follow HTTP redirects, configurable retries on NoHttpResponseException, compression, and not creating a new HttpClient on each request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-462) Performance related enhancements to
Solrj
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan McKinley resolved SOLR-462.
--------------------------------
Resolution: Fixed
Fix Version/s: 1.3
Assignee: Ryan McKinley
Wow! this is great. I committed this with a few style changes and changed the HttpClient creation to something overrideable.
Any reason _allowCompression should be false by default? Perhaps we should change it to true?
Thanks Sean!
> Performance related enhancements to Solrj
> -----------------------------------------
>
> Key: SOLR-462
> URL: https://issues.apache.org/jira/browse/SOLR-462
> Project: Solr
> Issue Type: New Feature
> Components: clients - java
> Reporter: Sean Timm
> Assignee: Ryan McKinley
> Fix For: 1.3
>
> Attachments: solrj-SOLR-462.patch
>
>
> Changes to CommonsHttpSolrServer.java to add soTimeout (read timeout), connection pool timeout, directive to not follow HTTP redirects, configurable retries on NoHttpResponseException, compression, and not creating a new HttpClient on each request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-462) Performance related enhancements to
Solrj
Posted by "Sean Timm (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561035#action_12561035 ]
Sean Timm commented on SOLR-462:
--------------------------------
There are two reasons I chose to not have compression enabled by default: (1) it seems like most Solr deployments use Tomcat or Jetty stand-alone--to get compression, you need to either use a servlet filter to do the compression or put the servlet container behind Apache or another web server enabled to provide the compression, and (2) if both the Solr servers and the Solrj clients are on the same local network, with a 100 Mbps or even gigabit connection, is the CPU overhead of compression going to buy you enough on the IO side? Now if you have Solr in one data center and your Solrj in a different data center, compression is likely a benefit. The option is nice and it is a quick thing to try to see if it helps in your particular circumstance.
> Performance related enhancements to Solrj
> -----------------------------------------
>
> Key: SOLR-462
> URL: https://issues.apache.org/jira/browse/SOLR-462
> Project: Solr
> Issue Type: New Feature
> Components: clients - java
> Reporter: Sean Timm
> Assignee: Ryan McKinley
> Fix For: 1.3
>
> Attachments: solrj-SOLR-462.patch
>
>
> Changes to CommonsHttpSolrServer.java to add soTimeout (read timeout), connection pool timeout, directive to not follow HTTP redirects, configurable retries on NoHttpResponseException, compression, and not creating a new HttpClient on each request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-462) Performance related enhancements to
Solrj
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561067#action_12561067 ]
Ryan McKinley commented on SOLR-462:
------------------------------------
I totally agree with why you would not want to use compression for the most normal use case -- connecting within a LAN or localhost. But If I understand correctly. From the client side the only difference is that the client says "hey, I can use compressed data if you give it to me" -- The *server* decides if compression is ok or not. In any normal case (using solr.war without editing web.xml and not going through a caching proxy) the results are not compressed even if the client says it is ok with compressed results.
a trivial point, and I'm happy to go with whatever you think is best....
> Performance related enhancements to Solrj
> -----------------------------------------
>
> Key: SOLR-462
> URL: https://issues.apache.org/jira/browse/SOLR-462
> Project: Solr
> Issue Type: New Feature
> Components: clients - java
> Reporter: Sean Timm
> Assignee: Ryan McKinley
> Fix For: 1.3
>
> Attachments: solrj-SOLR-462.patch
>
>
> Changes to CommonsHttpSolrServer.java to add soTimeout (read timeout), connection pool timeout, directive to not follow HTTP redirects, configurable retries on NoHttpResponseException, compression, and not creating a new HttpClient on each request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.