You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Ryan Yacyshyn <ry...@gmail.com> on 2018/02/21 02:54:15 UTC
Streaming Expressions using Solrj.io
Hello all,
I'd like to get a stream of search results using the solrj.io package but
running into a small issue. It seems to have something to do with the
HttpClientUtil. I'm testing on SolrCloud 7.1.0, using the
sample_techproducts_configs configs, and indexed the manufacturers.xml file.
I'm following the test code in the method `testCloudSolrStreamWithZkHost`
found in StreamExpressionTest.java:
```
package ca.ryac.testing;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.solr.client.solrj.io.SolrClientCache;
import org.apache.solr.client.solrj.io.Tuple;
import org.apache.solr.client.solrj.io.stream.CloudSolrStream;
import org.apache.solr.client.solrj.io.stream.StreamContext;
import org.apache.solr.client.solrj.io.stream.TupleStream;
import org.apache.solr.client.solrj.io.stream.expr.StreamExpression;
import org.apache.solr.client.solrj.io.stream.expr.StreamExpressionParser;
import org.apache.solr.client.solrj.io.stream.expr.StreamFactory;
public class SolrStreamingClient {
String zkHost = "localhost:9983";
String COLLECTIONORALIAS = "gettingstarted";
public SolrStreamingClient() throws Exception {
init();
}
public static void main(String[] args) throws Exception {
new SolrStreamingClient();
}
private void init() throws Exception {
System.out.println(zkHost);
StreamFactory factory = new StreamFactory();
StreamExpression expression;
CloudSolrStream stream;
List<Tuple> tuples;
StreamContext streamContext = new StreamContext();
SolrClientCache solrClientCache = new SolrClientCache();
streamContext.setSolrClientCache(solrClientCache);
// basic test..
String expr = "search(" + COLLECTIONORALIAS + ", zkHost=\"" + zkHost
+ "\", q=*:*, fl=\"id,compName_s\", sort=\"compName_s asc\")";
System.out.println(expr);
expression = StreamExpressionParser.parse(expr);
stream = new CloudSolrStream(expression, factory);
stream.setStreamContext(streamContext);
tuples = getTuples(stream);
System.out.println(tuples.size());
}
protected List<Tuple> getTuples(TupleStream tupleStream) throws
IOException {
List<Tuple> tuples = new ArrayList<Tuple>();
try {
System.out.println("open stream..");
tupleStream.open();
for (Tuple t = tupleStream.read(); !t.EOF; t = tupleStream.read()) {
tuples.add(t);
}
} finally {
tupleStream.close();
}
return tuples;
}
}
```
And this is the output I get:
---
localhost:9983
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
search(gettingstarted, zkHost="localhost:9983", q=*:*, fl="id,compName_s",
sort="compName_s asc")
open stream..
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.http.impl.client.HttpClientBuilder.evictIdleConnections(JLjava/util/concurrent/TimeUnit;)Lorg/apache/http/impl/client/HttpClientBuilder;
at
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:279)
at
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:298)
at
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:236)
at
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:223)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.<init>(CloudSolrClient.java:276)
at
org.apache.solr.client.solrj.impl.CloudSolrClient$Builder.build(CloudSolrClient.java:1525)
at
org.apache.solr.client.solrj.io.SolrClientCache.getCloudSolrClient(SolrClientCache.java:62)
at
org.apache.solr.client.solrj.io.stream.TupleStream.getShards(TupleStream.java:138)
at
org.apache.solr.client.solrj.io.stream.CloudSolrStream.constructStreams(CloudSolrStream.java:368)
at
org.apache.solr.client.solrj.io.stream.CloudSolrStream.open(CloudSolrStream.java:274)
at
ca.ryac.testing.SolrStreamingClient.getTuples(SolrStreamingClient.java:61)
at ca.ryac.testing.SolrStreamingClient.init(SolrStreamingClient.java:51)
at ca.ryac.testing.SolrStreamingClient.<init>(SolrStreamingClient.java:22)
at ca.ryac.testing.SolrStreamingClient.main(SolrStreamingClient.java:26)
---
It's not finding or connecting to my SolrCloud instance, I can put
*anything* in zkHost and get the same results. Not really sure why it can't
find or connect to it. Any thoughts or ideas?
Thank you,
Ryan
Re: Streaming Expressions using Solrj.io
Posted by Ryan Yacyshyn <ry...@gmail.com>.
Hi Shawn,
Thanks for this.
I'm using maven and I had another library unrelated to solrj loading an
older version of the httpclient (4.4.1), which was causing the problem.
Regards,
Ryan
On Wed, 21 Feb 2018 at 12:50 Shawn Heisey <ap...@elyograg.org> wrote:
> On 2/20/2018 7:54 PM, Ryan Yacyshyn wrote:
> > I'd like to get a stream of search results using the solrj.io package
> but
> > running into a small issue.
>
> <snip>
>
> > Exception in thread "main" java.lang.NoSuchMethodError:
> >
> org.apache.http.impl.client.HttpClientBuilder.evictIdleConnections(JLjava/util/concurrent/TimeUnit;)Lorg/apache/http/impl/client/HttpClientBuilder;
>
> There is a problem accessing the HttpClient library. Either the
> httpclient jar is missing from your project, or it's the wrong version.
> You can use pretty much any 4.5.x version for recent SolrJ versions.
> 3.x versions won't work at all, and older 4.x versions not work. The
> 5.0 beta releases also won't work. You can find information about 4.0
> and later versions of HttpClient here:
>
> http://hc.apache.org/
>
> If you use a dependency manager like gradle, maven, or ivy for your
> project, just be sure it's set to pull in all transitive dependencies
> for solrj, and you should be fine. If you manage dependencies manually,
> you will find all of the extra jars required by the solrj client in the
> download, in the dist/solrj-lib directory. Note that you can very
> likely upgrade individual dependencies to newer versions than Solr
> includes with no issues.
>
> Thanks,
> Shawn
>
>
Re: Streaming Expressions using Solrj.io
Posted by Shawn Heisey <ap...@elyograg.org>.
On 2/20/2018 7:54 PM, Ryan Yacyshyn wrote:
> I'd like to get a stream of search results using the solrj.io package but
> running into a small issue.
<snip>
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.apache.http.impl.client.HttpClientBuilder.evictIdleConnections(JLjava/util/concurrent/TimeUnit;)Lorg/apache/http/impl/client/HttpClientBuilder;
There is a problem accessing the HttpClient library. Either the
httpclient jar is missing from your project, or it's the wrong version.
You can use pretty much any 4.5.x version for recent SolrJ versions.
3.x versions won't work at all, and older 4.x versions not work. The
5.0 beta releases also won't work. You can find information about 4.0
and later versions of HttpClient here:
http://hc.apache.org/
If you use a dependency manager like gradle, maven, or ivy for your
project, just be sure it's set to pull in all transitive dependencies
for solrj, and you should be fine. If you manage dependencies manually,
you will find all of the extra jars required by the solrj client in the
download, in the dist/solrj-lib directory. Note that you can very
likely upgrade individual dependencies to newer versions than Solr
includes with no issues.
Thanks,
Shawn