You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Aman Deep Singh <am...@gmail.com> on 2017/06/16 06:17:53 UTC
Possible bug in Solrj-6.6.0
Hi,
I think their is a possible bug in Solrj version 6.6.0 ,as streaming is not
working
as i have a piece of code
public Set<String> getAllIds(String requestId, String field) {
LOG.info("Now Trying to fetch all the ids from SOLR for request Id
{}", requestId);
Map props = new HashMap();
props.put("q", field + ":*");
props.put("qt", "/export");
props.put("sort", field + " asc");
props.put("fl", field);
Set<String> idSet = new HashSet<>();
try (CloudSolrStream cloudSolrStream = new
CloudSolrStream(cloudSolrClient.getZkHost(),
cloudSolrClient.getDefaultCollection(), new MapSolrParams(props))) {
cloudSolrStream.open();
while (true) {
Tuple tuple = cloudSolrStream.read();
if (tuple.EOF) {
break;
}
idSet.add(tuple.getString(field));
}
return idSet;
} catch (IOException ex) {
LOG.error("Error while fetching the ids from SOLR for request
Id {} ", requestId, ex);
}
return Collections.emptySet();
}
This is working in the Solrj 6.5.1 but now it start throwing Error
after upgrading to solrj-6.6.0
java.io.IOException: java.lang.NullPointerException
at org.apache.solr.client.solrj.io.stream.CloudSolrStream.constructStreams(CloudSolrStream.java:408)
~[solr-solrj-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241
- ishan - 2017-05-30 07:32:54]
at org.apache.solr.client.solrj.io.stream.CloudSolrStream.open(CloudSolrStream.java:299)
~[solr-solrj-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241
- ishan - 2017-05-30 07:32:54]
Thanks,
Aman Deep Singh
Re: Possible bug in Solrj-6.6.0
Posted by Joel Bernstein <jo...@gmail.com>.
Yes that is correct.
Joel Bernstein
http://joelsolr.blogspot.com/
On Fri, Jun 16, 2017 at 9:55 AM, Aman Deep Singh <am...@gmail.com>
wrote:
> Thanks Joel,
> It is working now
> One quick question,as you say that we can use solr client cache multiple
> time so can I create a single instance of solr client cache and use it
> again and again ,since we are using one single bean for client object.
>
>
> On 16-Jun-2017 6:28 PM, "Joel Bernstein" <jo...@gmail.com> wrote:
>
> The issue is that in 6.6 CloudSolrStream is expecting a StreamContext to be
> set. So you'll need to update your code to do this. This was part of
> changes made to make streaming work in non-SolrCloud environments.
>
> You also need to create a SolrClientCache which caches the SolrClients.
>
> Example:
>
> SolrClientCache cache = new SolrClientCache();
>
> StreamContext streamContext = new StreamContext();
>
> streamContext.setSolrClientCache(cache);
>
> CloudSolrStream stream = new CloudSolrStream(...);
> stream.setStreamContext(streamContext);
> stream.open();
> ....
>
> The SolrClientCache can be shared by multiple requests and should be closed
> when the application exits.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Fri, Jun 16, 2017 at 2:17 AM, Aman Deep Singh <
> amandeep.cool99@gmail.com>
> wrote:
>
> > Hi,
> > I think their is a possible bug in Solrj version 6.6.0 ,as streaming is
> not
> > working
> > as i have a piece of code
> >
> > public Set<String> getAllIds(String requestId, String field) {
> > LOG.info("Now Trying to fetch all the ids from SOLR for request Id
> > {}", requestId);
> > Map props = new HashMap();
> > props.put("q", field + ":*");
> > props.put("qt", "/export");
> > props.put("sort", field + " asc");
> > props.put("fl", field);
> > Set<String> idSet = new HashSet<>();
> > try (CloudSolrStream cloudSolrStream = new
> > CloudSolrStream(cloudSolrClient.getZkHost(),
> > cloudSolrClient.getDefaultCollection(), new
> > MapSolrParams(props))) {
> > cloudSolrStream.open();
> > while (true) {
> > Tuple tuple = cloudSolrStream.read();
> > if (tuple.EOF) {
> > break;
> > }
> > idSet.add(tuple.getString(field));
> > }
> > return idSet;
> > } catch (IOException ex) {
> > LOG.error("Error while fetching the ids from SOLR for request
> > Id {} ", requestId, ex);
> > }
> > return Collections.emptySet();
> > }
> >
> >
> > This is working in the Solrj 6.5.1 but now it start throwing Error
> > after upgrading to solrj-6.6.0
> >
> > java.io.IOException: java.lang.NullPointerException
> > at org.apache.solr.client.solrj.io.stream.CloudSolrStream.
> > constructStreams(CloudSolrStream.java:408)
> > ~[solr-solrj-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241
> > - ishan - 2017-05-30 07:32:54]
> > at org.apache.solr.client.solrj.io.stream.CloudSolrStream.
> > open(CloudSolrStream.java:299)
> > ~[solr-solrj-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241
> > - ishan - 2017-05-30 07:32:54]
> >
> >
> > Thanks,
> >
> > Aman Deep Singh
> >
>
Re: Possible bug in Solrj-6.6.0
Posted by Aman Deep Singh <am...@gmail.com>.
Thanks Joel,
It is working now
One quick question,as you say that we can use solr client cache multiple
time so can I create a single instance of solr client cache and use it
again and again ,since we are using one single bean for client object.
On 16-Jun-2017 6:28 PM, "Joel Bernstein" <jo...@gmail.com> wrote:
The issue is that in 6.6 CloudSolrStream is expecting a StreamContext to be
set. So you'll need to update your code to do this. This was part of
changes made to make streaming work in non-SolrCloud environments.
You also need to create a SolrClientCache which caches the SolrClients.
Example:
SolrClientCache cache = new SolrClientCache();
StreamContext streamContext = new StreamContext();
streamContext.setSolrClientCache(cache);
CloudSolrStream stream = new CloudSolrStream(...);
stream.setStreamContext(streamContext);
stream.open();
....
The SolrClientCache can be shared by multiple requests and should be closed
when the application exits.
Joel Bernstein
http://joelsolr.blogspot.com/
On Fri, Jun 16, 2017 at 2:17 AM, Aman Deep Singh <am...@gmail.com>
wrote:
> Hi,
> I think their is a possible bug in Solrj version 6.6.0 ,as streaming is
not
> working
> as i have a piece of code
>
> public Set<String> getAllIds(String requestId, String field) {
> LOG.info("Now Trying to fetch all the ids from SOLR for request Id
> {}", requestId);
> Map props = new HashMap();
> props.put("q", field + ":*");
> props.put("qt", "/export");
> props.put("sort", field + " asc");
> props.put("fl", field);
> Set<String> idSet = new HashSet<>();
> try (CloudSolrStream cloudSolrStream = new
> CloudSolrStream(cloudSolrClient.getZkHost(),
> cloudSolrClient.getDefaultCollection(), new
> MapSolrParams(props))) {
> cloudSolrStream.open();
> while (true) {
> Tuple tuple = cloudSolrStream.read();
> if (tuple.EOF) {
> break;
> }
> idSet.add(tuple.getString(field));
> }
> return idSet;
> } catch (IOException ex) {
> LOG.error("Error while fetching the ids from SOLR for request
> Id {} ", requestId, ex);
> }
> return Collections.emptySet();
> }
>
>
> This is working in the Solrj 6.5.1 but now it start throwing Error
> after upgrading to solrj-6.6.0
>
> java.io.IOException: java.lang.NullPointerException
> at org.apache.solr.client.solrj.io.stream.CloudSolrStream.
> constructStreams(CloudSolrStream.java:408)
> ~[solr-solrj-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241
> - ishan - 2017-05-30 07:32:54]
> at org.apache.solr.client.solrj.io.stream.CloudSolrStream.
> open(CloudSolrStream.java:299)
> ~[solr-solrj-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241
> - ishan - 2017-05-30 07:32:54]
>
>
> Thanks,
>
> Aman Deep Singh
>
Re: Possible bug in Solrj-6.6.0
Posted by Joel Bernstein <jo...@gmail.com>.
The issue is that in 6.6 CloudSolrStream is expecting a StreamContext to be
set. So you'll need to update your code to do this. This was part of
changes made to make streaming work in non-SolrCloud environments.
You also need to create a SolrClientCache which caches the SolrClients.
Example:
SolrClientCache cache = new SolrClientCache();
StreamContext streamContext = new StreamContext();
streamContext.setSolrClientCache(cache);
CloudSolrStream stream = new CloudSolrStream(...);
stream.setStreamContext(streamContext);
stream.open();
....
The SolrClientCache can be shared by multiple requests and should be closed
when the application exits.
Joel Bernstein
http://joelsolr.blogspot.com/
On Fri, Jun 16, 2017 at 2:17 AM, Aman Deep Singh <am...@gmail.com>
wrote:
> Hi,
> I think their is a possible bug in Solrj version 6.6.0 ,as streaming is not
> working
> as i have a piece of code
>
> public Set<String> getAllIds(String requestId, String field) {
> LOG.info("Now Trying to fetch all the ids from SOLR for request Id
> {}", requestId);
> Map props = new HashMap();
> props.put("q", field + ":*");
> props.put("qt", "/export");
> props.put("sort", field + " asc");
> props.put("fl", field);
> Set<String> idSet = new HashSet<>();
> try (CloudSolrStream cloudSolrStream = new
> CloudSolrStream(cloudSolrClient.getZkHost(),
> cloudSolrClient.getDefaultCollection(), new
> MapSolrParams(props))) {
> cloudSolrStream.open();
> while (true) {
> Tuple tuple = cloudSolrStream.read();
> if (tuple.EOF) {
> break;
> }
> idSet.add(tuple.getString(field));
> }
> return idSet;
> } catch (IOException ex) {
> LOG.error("Error while fetching the ids from SOLR for request
> Id {} ", requestId, ex);
> }
> return Collections.emptySet();
> }
>
>
> This is working in the Solrj 6.5.1 but now it start throwing Error
> after upgrading to solrj-6.6.0
>
> java.io.IOException: java.lang.NullPointerException
> at org.apache.solr.client.solrj.io.stream.CloudSolrStream.
> constructStreams(CloudSolrStream.java:408)
> ~[solr-solrj-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241
> - ishan - 2017-05-30 07:32:54]
> at org.apache.solr.client.solrj.io.stream.CloudSolrStream.
> open(CloudSolrStream.java:299)
> ~[solr-solrj-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241
> - ishan - 2017-05-30 07:32:54]
>
>
> Thanks,
>
> Aman Deep Singh
>