You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Scott Johnson <sj...@dag.com> on 2014/10/03 21:08:20 UTC
Inconsistent response time
We are attempting to improve our Solr response time as our application uses
Solr for large and time consuming queries. We have found a very inconsistent
result in the time elapsed when pinging Solr. If we ping Solr from a desktop
Windows 7 machine, there is usually a 5 ms elapsed time. But if we ping the
same Solr instance from a Windows Server 2008 machine, it takes about 15 ms.
This could be the difference between a 1 hour process and a 3 hour process,
so it is something we would like to debug and fix if possible.
Does anybody have any ideas about why this might be? We get these same
results pretty consistently (testing on multiple desktops and servers). One
thing that seemed to have an impact is removing various additional JDKs that
had been installed, and JDK 1.7u67 specifically seemed to make a difference.
Finally, the code we are suing to test this is below. If there is a better
test I would be curious to hear that as well.
Thanks,
Scott
package solr;
import org.apache.commons.lang.StringUtils;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrRequest.METHOD;
import org.apache.solr.client.solrj.impl.BinaryRequestWriter;
import org.apache.solr.client.solrj.impl.BinaryResponseParser;
import org.apache.solr.client.solrj.impl.HttpSolrServer;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.client.solrj.response.SolrPingResponse;
import org.apache.solr.common.SolrDocumentList;
public class SolrTest {
private HttpSolrServer server;
/**
* @param args
* @throws Exception
*/
public static void main(String[] args) throws Exception {
SolrTest solr = new SolrTest(args);
// Run it a few times, the second time runs
a lot faster.
for (int i=0; i<3; i++) {
solr.execute();
}
}
public SolrTest(String[] args) throws Exception {
String targetUrl = args[0];
System.out.println("=============System
properties=============");
System.out.println("Start solr test.... " +
targetUrl);
server = new HttpSolrServer("http://" +
targetUrl + ":8111/solr/search/");
server.setRequestWriter(new
BinaryRequestWriter());
server.setParser(new
BinaryResponseParser());
server.setAllowCompression(true);
server.setDefaultMaxConnectionsPerHost(128);
server.setMaxTotalConnections(128);
SolrPingResponse response = server.ping();
System.out.println("Ping time: " +
response.getElapsedTime() + " ms");
System.out.println("Ping time: " +
response.getElapsedTime() + " ms");
}
private void execute() throws Exception {
SolrQuery query = new SolrQuery();
query.setParam("start", "0");
query.setParam("rows", "1");
long startTime = System.currentTimeMillis();
QueryResponse queryResponse =
server.query(query, METHOD.POST);
long elapsedTime =
(System.currentTimeMillis() - startTime);
SolrDocumentList results =
queryResponse.getResults();
long totalHits = results.getNumFound();
System.out.println("Search hits:" +
totalHits
+ ". Total
elapsed time:" + elapsedTime + " ms"
+ ". Solr
elapsed time:" + queryResponse.getElapsedTime() + " ms"
+ ". Solr
query time:" + queryResponse.getQTime() + " ms"
+ ". Params:
" + getSearchParams(query));
}
/**
* Formats solr query parameters so that we know what's passed to solr.
* @param query
* @return
*/
private String getSearchParams(SolrQuery query) {
StringBuilder sb = new StringBuilder();
boolean first = true;
for (String name :
query.getParameterNames()) {
if (first) {
first =
false;
} else {
sb.append(";
");
}
sb.append(name).append(":[").append(StringUtils.join(query.getParams(name),
", ")).append("]");
}
return sb.toString();
}
}
Re: Inconsistent response time
Posted by Chris Hostetter <ho...@fucit.org>.
: Solr for large and time consuming queries. We have found a very inconsistent
: result in the time elapsed when pinging Solr. If we ping Solr from a desktop
: Windows 7 machine, there is usually a 5 ms elapsed time. But if we ping the
: same Solr instance from a Windows Server 2008 machine, it takes about 15 ms.
: Does anybody have any ideas about why this might be? We get these same
: results pretty consistently (testing on multiple desktops and servers). One
the devil is really in the details here ... and you haven't provided very
many.
Define "multiple desktops and servers" ... do you specifically mean
multiple *local* desktop machines as the client and multiple *remote*
servers running solr, or are you also refering to "windows server as
client talking solr server" situations? ("server is vague and ambiguious
in your email, particularly since one of hte variables is "Windows Server
2008")
what is your net architecture?
how/where are all of the various machines physically located/connected on
the network?
Hypothetical cause of your problem that wold fit all of the info you have
provided: your solr server is a test machine sitting hte same building as
all of your "desktop" Window 7 machines, and so it's just a local ethernet
hop away, but your "Windows Server 2008" machine is a "server" in some
remote data center and has more network hops to go through.
see what i mean about hte details mattering?
when you run this code, is the command line arg you specify an IP addr o a
hostname? have you ruled out DNS lookup discrepencies between the diff
client operating systems?
: thing that seemed to have an impact is removing various additional JDKs that
: had been installed, and JDK 1.7u67 specifically seemed to make a difference.
impact how? Does JDK 1.7u67 make your tests go faster or slower? faster
or slower then what? what other java versions did you try?
: Finally, the code we are suing to test this is below. If there is a better
: test I would be curious to hear that as well.
well, since the problem you are describing doesn't seem to have anything
to do with solr, removing solr & solrj completely from the equation would
be the first thing i would test.
do you have a "curl" equivilent on these windows machines? have you tried
just fetching the ping URL w/o using the SolrJ code and comparing hte
response times that way?
the XML format of the ping response is also really trivial -- you could
save it to a plain XML file, toss it up in a directory on your favorite
webserver, and point either curl (or the solrj SolrServer class) at that
to sanity check wether anything in Solr (or the solrj SolrServer class) is
actually causing the discrepency you are seeing.
My best guess is the discrepencies you are seeing are entirely network/os
based and have nothing to do with solr.
-Hoss
http://www.lucidworks.com/
RE: Inconsistent response time
Posted by Michael Ryan <mr...@moreover.com>.
It could be due to the minimum timer resolution on Windows. Do a search for "windows 15ms" and you'll find a lot of information about it. Though, I'm not sure which versions of Windows and/or Java have that problem. You could test it out by timing things other than Solr and see if they also take 15ms. I often see 15ms, 16ms, 31ms, and 32ms when timing stuff on Windows.
-Michael
-----Original Message-----
From: Scott Johnson [mailto:sjohnson@dag.com]
Sent: Friday, October 03, 2014 5:58 PM
To: solr-user@lucene.apache.org
Subject: RE: Inconsistent response time
Thanks for the recommendation, but that is not making a difference here.
-----Original Message-----
From: Michael Della Bitta [mailto:michael.della.bitta@appinions.com]
Sent: Friday, October 03, 2014 2:00 PM
To: solr-user@lucene.apache.org
Subject: Re: Inconsistent response time
Hi Scott,
Any chance this could be an IPv6 thing? What if you start both server and client with this flag:
-Djava.net.preferIPv4Stack=true
Michael Della Bitta
Senior Software Engineer
o: +1 646 532 3062
appinions inc.
"The Science of Influence Marketing"
18 East 41st Street
New York, NY 10017
t: @appinions | g+: plus.google.com/appinions
w: appinions.com
On Oct 3, 2014, at 15:08, Scott Johnson <sj...@dag.com> wrote:
> We are attempting to improve our Solr response time as our application
> uses Solr for large and time consuming queries. We have found a very
> inconsistent result in the time elapsed when pinging Solr. If we ping
> Solr from a desktop Windows 7 machine, there is usually a 5 ms elapsed
> time. But if we ping the same Solr instance from a Windows Server 2008
machine, it takes about 15 ms.
> This could be the difference between a 1 hour process and a 3 hour
> process, so it is something we would like to debug and fix if possible.
>
>
>
> Does anybody have any ideas about why this might be? We get these same
> results pretty consistently (testing on multiple desktops and
> servers). One thing that seemed to have an impact is removing various
> additional JDKs that had been installed, and JDK 1.7u67 specifically
seemed to make a difference.
>
>
>
> Finally, the code we are suing to test this is below. If there is a
> better test I would be curious to hear that as well.
>
>
>
> Thanks,
>
>
> Scott
>
>
>
>
>
> package solr;
>
>
>
> import org.apache.commons.lang.StringUtils;
>
> import org.apache.solr.client.solrj.SolrQuery;
>
> import org.apache.solr.client.solrj.SolrRequest.METHOD;
>
> import org.apache.solr.client.solrj.impl.BinaryRequestWriter;
>
> import org.apache.solr.client.solrj.impl.BinaryResponseParser;
>
> import org.apache.solr.client.solrj.impl.HttpSolrServer;
>
> import org.apache.solr.client.solrj.response.QueryResponse;
>
> import org.apache.solr.client.solrj.response.SolrPingResponse;
>
> import org.apache.solr.common.SolrDocumentList;
>
>
>
> public class SolrTest {
>
>
>
> private HttpSolrServer server;
>
>
>
> /**
>
> * @param args
>
> * @throws Exception
>
> */
>
> public static void main(String[] args) throws Exception
> {
>
> SolrTest solr = new SolrTest(args);
>
> // Run it a few times, the second time
> runs a lot faster.
>
> for (int i=0; i<3; i++) {
>
> solr.execute();
>
> }
>
> }
>
>
>
> public SolrTest(String[] args) throws Exception {
>
> String targetUrl = args[0];
>
>
>
> System.out.println("=============System
> properties=============");
>
> System.out.println("Start solr test....
> " + targetUrl);
>
>
>
> server = new HttpSolrServer("http://" +
> targetUrl + ":8111/solr/search/");
>
> server.setRequestWriter(new
> BinaryRequestWriter());
>
> server.setParser(new
> BinaryResponseParser());
>
> server.setAllowCompression(true);
>
>
> server.setDefaultMaxConnectionsPerHost(128);
>
> server.setMaxTotalConnections(128);
>
>
>
> SolrPingResponse response =
> server.ping();
>
> System.out.println("Ping time: " +
> response.getElapsedTime() + " ms");
>
> System.out.println("Ping time: " +
> response.getElapsedTime() + " ms");
>
> }
>
>
>
> private void execute() throws Exception {
>
> SolrQuery query = new SolrQuery();
>
> query.setParam("start", "0");
>
> query.setParam("rows", "1");
>
>
>
> long startTime =
> System.currentTimeMillis();
>
>
>
> QueryResponse queryResponse =
> server.query(query, METHOD.POST);
>
>
>
> long elapsedTime =
> (System.currentTimeMillis() - startTime);
>
>
>
> SolrDocumentList results =
> queryResponse.getResults();
>
> long totalHits = results.getNumFound();
>
>
>
> System.out.println("Search hits:" +
> totalHits
>
> + ".
> Total elapsed time:" + elapsedTime + " ms"
>
> + ".
> Solr elapsed time:" + queryResponse.getElapsedTime() + " ms"
>
> + ".
> Solr query time:" + queryResponse.getQTime() + " ms"
>
> + ".
Params:
> " + getSearchParams(query));
>
> }
>
>
>
>
>
> /**
>
> * Formats solr query parameters so that we know what's passed to solr.
>
> * @param query
>
> * @return
>
> */
>
> private String getSearchParams(SolrQuery query) {
>
> StringBuilder sb = new StringBuilder();
>
> boolean first = true;
>
>
>
> for (String name :
> query.getParameterNames()) {
>
> if (first) {
>
> first =
> false;
>
> } else {
>
>
> sb.append("; ");
>
> }
>
>
> sb.append(name).append(":[").append(StringUtils.join(query.getParams(n
> ame),
> ", ")).append("]");
>
> }
>
> return sb.toString();
>
> }
>
> }
>
RE: Inconsistent response time
Posted by Scott Johnson <sj...@dag.com>.
Thanks for the recommendation, but that is not making a difference here.
-----Original Message-----
From: Michael Della Bitta [mailto:michael.della.bitta@appinions.com]
Sent: Friday, October 03, 2014 2:00 PM
To: solr-user@lucene.apache.org
Subject: Re: Inconsistent response time
Hi Scott,
Any chance this could be an IPv6 thing? What if you start both server and
client with this flag:
-Djava.net.preferIPv4Stack=true
Michael Della Bitta
Senior Software Engineer
o: +1 646 532 3062
appinions inc.
"The Science of Influence Marketing"
18 East 41st Street
New York, NY 10017
t: @appinions | g+: plus.google.com/appinions
w: appinions.com
On Oct 3, 2014, at 15:08, Scott Johnson <sj...@dag.com> wrote:
> We are attempting to improve our Solr response time as our application
> uses Solr for large and time consuming queries. We have found a very
> inconsistent result in the time elapsed when pinging Solr. If we ping
> Solr from a desktop Windows 7 machine, there is usually a 5 ms elapsed
> time. But if we ping the same Solr instance from a Windows Server 2008
machine, it takes about 15 ms.
> This could be the difference between a 1 hour process and a 3 hour
> process, so it is something we would like to debug and fix if possible.
>
>
>
> Does anybody have any ideas about why this might be? We get these same
> results pretty consistently (testing on multiple desktops and
> servers). One thing that seemed to have an impact is removing various
> additional JDKs that had been installed, and JDK 1.7u67 specifically
seemed to make a difference.
>
>
>
> Finally, the code we are suing to test this is below. If there is a
> better test I would be curious to hear that as well.
>
>
>
> Thanks,
>
>
> Scott
>
>
>
>
>
> package solr;
>
>
>
> import org.apache.commons.lang.StringUtils;
>
> import org.apache.solr.client.solrj.SolrQuery;
>
> import org.apache.solr.client.solrj.SolrRequest.METHOD;
>
> import org.apache.solr.client.solrj.impl.BinaryRequestWriter;
>
> import org.apache.solr.client.solrj.impl.BinaryResponseParser;
>
> import org.apache.solr.client.solrj.impl.HttpSolrServer;
>
> import org.apache.solr.client.solrj.response.QueryResponse;
>
> import org.apache.solr.client.solrj.response.SolrPingResponse;
>
> import org.apache.solr.common.SolrDocumentList;
>
>
>
> public class SolrTest {
>
>
>
> private HttpSolrServer server;
>
>
>
> /**
>
> * @param args
>
> * @throws Exception
>
> */
>
> public static void main(String[] args) throws Exception
> {
>
> SolrTest solr = new SolrTest(args);
>
> // Run it a few times, the second time
> runs a lot faster.
>
> for (int i=0; i<3; i++) {
>
> solr.execute();
>
> }
>
> }
>
>
>
> public SolrTest(String[] args) throws Exception {
>
> String targetUrl = args[0];
>
>
>
> System.out.println("=============System
> properties=============");
>
> System.out.println("Start solr test....
> " + targetUrl);
>
>
>
> server = new HttpSolrServer("http://" +
> targetUrl + ":8111/solr/search/");
>
> server.setRequestWriter(new
> BinaryRequestWriter());
>
> server.setParser(new
> BinaryResponseParser());
>
> server.setAllowCompression(true);
>
>
> server.setDefaultMaxConnectionsPerHost(128);
>
> server.setMaxTotalConnections(128);
>
>
>
> SolrPingResponse response =
> server.ping();
>
> System.out.println("Ping time: " +
> response.getElapsedTime() + " ms");
>
> System.out.println("Ping time: " +
> response.getElapsedTime() + " ms");
>
> }
>
>
>
> private void execute() throws Exception {
>
> SolrQuery query = new SolrQuery();
>
> query.setParam("start", "0");
>
> query.setParam("rows", "1");
>
>
>
> long startTime =
> System.currentTimeMillis();
>
>
>
> QueryResponse queryResponse =
> server.query(query, METHOD.POST);
>
>
>
> long elapsedTime =
> (System.currentTimeMillis() - startTime);
>
>
>
> SolrDocumentList results =
> queryResponse.getResults();
>
> long totalHits = results.getNumFound();
>
>
>
> System.out.println("Search hits:" +
> totalHits
>
> + ".
> Total elapsed time:" + elapsedTime + " ms"
>
> + ".
> Solr elapsed time:" + queryResponse.getElapsedTime() + " ms"
>
> + ".
> Solr query time:" + queryResponse.getQTime() + " ms"
>
> + ".
Params:
> " + getSearchParams(query));
>
> }
>
>
>
>
>
> /**
>
> * Formats solr query parameters so that we know what's passed to solr.
>
> * @param query
>
> * @return
>
> */
>
> private String getSearchParams(SolrQuery query) {
>
> StringBuilder sb = new StringBuilder();
>
> boolean first = true;
>
>
>
> for (String name :
> query.getParameterNames()) {
>
> if (first) {
>
> first =
> false;
>
> } else {
>
>
> sb.append("; ");
>
> }
>
>
> sb.append(name).append(":[").append(StringUtils.join(query.getParams(n
> ame),
> ", ")).append("]");
>
> }
>
> return sb.toString();
>
> }
>
> }
>
Re: Inconsistent response time
Posted by Michael Della Bitta <mi...@appinions.com>.
Hi Scott,
Any chance this could be an IPv6 thing? What if you start both server and client with this flag:
-Djava.net.preferIPv4Stack=true
Michael Della Bitta
Senior Software Engineer
o: +1 646 532 3062
appinions inc.
“The Science of Influence Marketing”
18 East 41st Street
New York, NY 10017
t: @appinions | g+: plus.google.com/appinions
w: appinions.com
On Oct 3, 2014, at 15:08, Scott Johnson <sj...@dag.com> wrote:
> We are attempting to improve our Solr response time as our application uses
> Solr for large and time consuming queries. We have found a very inconsistent
> result in the time elapsed when pinging Solr. If we ping Solr from a desktop
> Windows 7 machine, there is usually a 5 ms elapsed time. But if we ping the
> same Solr instance from a Windows Server 2008 machine, it takes about 15 ms.
> This could be the difference between a 1 hour process and a 3 hour process,
> so it is something we would like to debug and fix if possible.
>
>
>
> Does anybody have any ideas about why this might be? We get these same
> results pretty consistently (testing on multiple desktops and servers). One
> thing that seemed to have an impact is removing various additional JDKs that
> had been installed, and JDK 1.7u67 specifically seemed to make a difference.
>
>
>
> Finally, the code we are suing to test this is below. If there is a better
> test I would be curious to hear that as well.
>
>
>
> Thanks,
>
>
> Scott
>
>
>
>
>
> package solr;
>
>
>
> import org.apache.commons.lang.StringUtils;
>
> import org.apache.solr.client.solrj.SolrQuery;
>
> import org.apache.solr.client.solrj.SolrRequest.METHOD;
>
> import org.apache.solr.client.solrj.impl.BinaryRequestWriter;
>
> import org.apache.solr.client.solrj.impl.BinaryResponseParser;
>
> import org.apache.solr.client.solrj.impl.HttpSolrServer;
>
> import org.apache.solr.client.solrj.response.QueryResponse;
>
> import org.apache.solr.client.solrj.response.SolrPingResponse;
>
> import org.apache.solr.common.SolrDocumentList;
>
>
>
> public class SolrTest {
>
>
>
> private HttpSolrServer server;
>
>
>
> /**
>
> * @param args
>
> * @throws Exception
>
> */
>
> public static void main(String[] args) throws Exception {
>
> SolrTest solr = new SolrTest(args);
>
> // Run it a few times, the second time runs
> a lot faster.
>
> for (int i=0; i<3; i++) {
>
> solr.execute();
>
> }
>
> }
>
>
>
> public SolrTest(String[] args) throws Exception {
>
> String targetUrl = args[0];
>
>
>
> System.out.println("=============System
> properties=============");
>
> System.out.println("Start solr test.... " +
> targetUrl);
>
>
>
> server = new HttpSolrServer("http://" +
> targetUrl + ":8111/solr/search/");
>
> server.setRequestWriter(new
> BinaryRequestWriter());
>
> server.setParser(new
> BinaryResponseParser());
>
> server.setAllowCompression(true);
>
> server.setDefaultMaxConnectionsPerHost(128);
>
> server.setMaxTotalConnections(128);
>
>
>
> SolrPingResponse response = server.ping();
>
> System.out.println("Ping time: " +
> response.getElapsedTime() + " ms");
>
> System.out.println("Ping time: " +
> response.getElapsedTime() + " ms");
>
> }
>
>
>
> private void execute() throws Exception {
>
> SolrQuery query = new SolrQuery();
>
> query.setParam("start", "0");
>
> query.setParam("rows", "1");
>
>
>
> long startTime = System.currentTimeMillis();
>
>
>
> QueryResponse queryResponse =
> server.query(query, METHOD.POST);
>
>
>
> long elapsedTime =
> (System.currentTimeMillis() - startTime);
>
>
>
> SolrDocumentList results =
> queryResponse.getResults();
>
> long totalHits = results.getNumFound();
>
>
>
> System.out.println("Search hits:" +
> totalHits
>
> + ". Total
> elapsed time:" + elapsedTime + " ms"
>
> + ". Solr
> elapsed time:" + queryResponse.getElapsedTime() + " ms"
>
> + ". Solr
> query time:" + queryResponse.getQTime() + " ms"
>
> + ". Params:
> " + getSearchParams(query));
>
> }
>
>
>
>
>
> /**
>
> * Formats solr query parameters so that we know what's passed to solr.
>
> * @param query
>
> * @return
>
> */
>
> private String getSearchParams(SolrQuery query) {
>
> StringBuilder sb = new StringBuilder();
>
> boolean first = true;
>
>
>
> for (String name :
> query.getParameterNames()) {
>
> if (first) {
>
> first =
> false;
>
> } else {
>
> sb.append(";
> ");
>
> }
>
>
> sb.append(name).append(":[").append(StringUtils.join(query.getParams(name),
> ", ")).append("]");
>
> }
>
> return sb.toString();
>
> }
>
> }
>