You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Satish Talim <sa...@gmail.com> on 2011/08/30 05:42:19 UTC
How to send an OpenBitSet object from Solr server?
We have a need to query and fetch millions of document ids from a Solr 3.3
index and convert the same to a BitSet. To speed things up, we want to
convert these document ids into OpenBitSet on the server side, put them into
the response object and read the same on the client side.
To achieve this, we wrote our own RequestHandler and overwrote
the handleRequest method. Using this RequestHandler we do get the response
object but when we try to fetch the OpenBitSet we get an error -
Exception in thread "main" java.lang.ClassCastException: java.lang.String
cannot be cast to org.apache.lucene.util.OpenBitSet
The documentation at -
http://lucene.apache.org/solr/api/org/apache/solr/response/SolrQueryResponse.html
says that "Other data types may be added to the SolrQueryResponse, but there
is no guarantee that QueryResponseWriters will be able to deal with
unexpected types."
Is there a work-around wherein I can send an OpenBitSet object?
Satish
Re: How to send an OpenBitSet object from Solr server?
Posted by Chris Hostetter <ho...@fucit.org>.
: I was not referring to Lucene's doc ids but the doc numbers (unique key)
Uh ... ok. this is why i asked you waht you were planning on doing with
the OpenBitSet -- it's just bits, indicating hte offsets of the documents
in the total index. having access to a copy of that on the lcient side
isn't going to give you any info about the value of the uniqueKey field.
Like i said ... XY Problem ... please tell us what you overall goal is.
why do you want the uniqueKey values for every matching doc in your remove
client? What would you do with the OpenBitSet from solr if you had a copy
of it?
: > : We have a need to query and fetch millions of document ids from a Solr
: > 3.3
: > : index and convert the same to a BitSet. To speed things up, we want to
: > : convert these document ids into OpenBitSet on the server side, put them
: > into
: > : the response object and read the same on the client side.
: >
: > This smells like an XY Problem ... what do you intend to do with this
: > BitSet on the client side? the lucene doc ids are meaningless outside of
: > hte server, and for any given doc, the id could change from one request to
: > the next -- so how would having this data on the clinet be of any use to
: > you?
: >
: > https://people.apache.org/~hossman/#xyproblem
: > XY Problem
: >
: > Your question appears to be an "XY Problem" ... that is: you are dealing
: > with "X", you are assuming "Y" will help you, and you are asking about "Y"
: > without giving more details about the "X" so that we can understand the
: > full issue. Perhaps the best solution doesn't involve "Y" at all?
: > See Also: http://www.perlmonks.org/index.pl?node_id=542341
-Hoss
Re: How to send an OpenBitSet object from Solr server?
Posted by Satish Talim <sa...@gmail.com>.
I was not referring to Lucene's doc ids but the doc numbers (unique key)
Satish
On Tue, Aug 30, 2011 at 9:28 PM, Chris Hostetter
<ho...@fucit.org>wrote:
>
> : We have a need to query and fetch millions of document ids from a Solr
> 3.3
> : index and convert the same to a BitSet. To speed things up, we want to
> : convert these document ids into OpenBitSet on the server side, put them
> into
> : the response object and read the same on the client side.
>
> This smells like an XY Problem ... what do you intend to do with this
> BitSet on the client side? the lucene doc ids are meaningless outside of
> hte server, and for any given doc, the id could change from one request to
> the next -- so how would having this data on the clinet be of any use to
> you?
>
> https://people.apache.org/~hossman/#xyproblem
> XY Problem
>
> Your question appears to be an "XY Problem" ... that is: you are dealing
> with "X", you are assuming "Y" will help you, and you are asking about "Y"
> without giving more details about the "X" so that we can understand the
> full issue. Perhaps the best solution doesn't involve "Y" at all?
> See Also: http://www.perlmonks.org/index.pl?node_id=542341
>
>
> -Hoss
>
Re: How to send an OpenBitSet object from Solr server?
Posted by Chris Hostetter <ho...@fucit.org>.
: We have a need to query and fetch millions of document ids from a Solr 3.3
: index and convert the same to a BitSet. To speed things up, we want to
: convert these document ids into OpenBitSet on the server side, put them into
: the response object and read the same on the client side.
This smells like an XY Problem ... what do you intend to do with this
BitSet on the client side? the lucene doc ids are meaningless outside of
hte server, and for any given doc, the id could change from one request to
the next -- so how would having this data on the clinet be of any use to
you?
https://people.apache.org/~hossman/#xyproblem
XY Problem
Your question appears to be an "XY Problem" ... that is: you are dealing
with "X", you are assuming "Y" will help you, and you are asking about "Y"
without giving more details about the "X" so that we can understand the
full issue. Perhaps the best solution doesn't involve "Y" at all?
See Also: http://www.perlmonks.org/index.pl?node_id=542341
-Hoss
Re: How to send an OpenBitSet object from Solr server?
Posted by Federico Fissore <fi...@celi.it>.
Satish Talim, il 30/08/2011 14:22, ha scritto:
> But how to throw? As a stream of bits?
>
getBits() return a long[]
add a long[] part to your response
rb.rsp.add("long_array", obs.getBits())
federico
Re: How to send an OpenBitSet object from Solr server?
Posted by Satish Talim <sa...@gmail.com>.
But how to throw? As a stream of bits?
Satish
On Tue, Aug 30, 2011 at 5:39 PM, Federico Fissore <fe...@fissore.org>wrote:
> Satish Talim, il 30/08/2011 05:42, ha scritto:
> [...]
>
>
>> Is there a work-around wherein I can send an OpenBitSet object?
>>
>>
> JavaBinCodec (used by default by solr) supports writing arrays. you can
> "getBits()" from openbitset and throw them into the binary response
>
> federico
>
Re: How to send an OpenBitSet object from Solr server?
Posted by Federico Fissore <fe...@fissore.org>.
Satish Talim, il 30/08/2011 05:42, ha scritto:
[...]
>
> Is there a work-around wherein I can send an OpenBitSet object?
>
JavaBinCodec (used by default by solr) supports writing arrays. you can
"getBits()" from openbitset and throw them into the binary response
federico
Re: How to send an OpenBitSet object from Solr server?
Posted by Federico Fissore <fi...@celi.it>.
Satish Talim, il 30/08/2011 05:42, ha scritto:
[...]
>
> Is there a work-around wherein I can send an OpenBitSet object?
>
JavaBinCodec (used by default by solr) supports writing arrays. you can
"getBits()" from openbitset and throw them into the binary response
federico