You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Barbet Alain <al...@gmail.com> on 2017/08/11 13:41:55 UTC
Fetch a binary field
Hi !
I've a Lucene base coming from a C++ program linked with Lucene++, a
port of Lucene 3.5.9. When I open this base with Luke, it show Lucene
2.9. Can see a binary field I have in Luke, with data encoded in
base64.
I have upgrade this base from 2.9 => 4.0 => 5.0 =>6.0 so I can use it
with Solr 6.5.1. I rebuild a schema for this base, who have one field
in binary:
<field name="document" type="binary" indexed="false" stored="true"/>
When I try to retrieve this field with a query (with SOLR admin
interface) it's fail with "can not use FieldCache on a field which is
neither indexed nor has doc values: document"
I can retrieve others fields, but can't find a way for this one. Does
someone have an idea ? (At the end I want do this with php, but as it
fail with Solr interface too ....).
Thank you for any help !
Re: Fetch a binary field
Posted by Barbet Alain <al...@gmail.com>.
Mmmmh ... my problem come from another side.
My field name has ":" in it. To avoid this, i use this in request:
fl=(field(name:point) but it's look like it isn't just to escape the
field name, it's do stuff on it.
If I use
fl=name*point
no problem I get my binary data
2017-08-18 6:01 GMT+02:00 GW <th...@gmail.com>:
> Had the same issue with long base64_encoded images. Binary & string failed.
> Set my field type to field type ignored. Doesn't seem right (or wrong) but
> it worked.
>
> On 17 August 2017 at 03:58, Rick Leir <rl...@leirtech.com> wrote:
>
>> On 2017-08-12 04:19 AM, Barbet Alain wrote:
>>
>>> Hi !
>>>
>>> Because this field containt a zipped xml that is bigger than all
>>> others fields & I don't need it for searching, just for display. Yes
>>> it would be better if this field is outside the Lucene base, but as I
>>> have hundred of bases like that, with millions of documents for each,
>>> no I can't change this & reindex the stuff ...
>>>
>>> Any other idea ?
>>>
>> Alain,
>> Since nobody else said it, after a long while...
>> Your zipped xml could be opened before indexing. You should just index the
>> data from the xml which will be needed for display. Hopefully that will
>> consume less index space than the whole xml.
>> cheers -- Rick
>>
Re: Fetch a binary field
Posted by GW <th...@gmail.com>.
Had the same issue with long base64_encoded images. Binary & string failed.
Set my field type to field type ignored. Doesn't seem right (or wrong) but
it worked.
On 17 August 2017 at 03:58, Rick Leir <rl...@leirtech.com> wrote:
> On 2017-08-12 04:19 AM, Barbet Alain wrote:
>
>> Hi !
>>
>> Because this field containt a zipped xml that is bigger than all
>> others fields & I don't need it for searching, just for display. Yes
>> it would be better if this field is outside the Lucene base, but as I
>> have hundred of bases like that, with millions of documents for each,
>> no I can't change this & reindex the stuff ...
>>
>> Any other idea ?
>>
> Alain,
> Since nobody else said it, after a long while...
> Your zipped xml could be opened before indexing. You should just index the
> data from the xml which will be needed for display. Hopefully that will
> consume less index space than the whole xml.
> cheers -- Rick
>
Re: Fetch a binary field
Posted by Rick Leir <rl...@leirtech.com>.
On 2017-08-12 04:19 AM, Barbet Alain wrote:
> Hi !
>
> Because this field containt a zipped xml that is bigger than all
> others fields & I don't need it for searching, just for display. Yes
> it would be better if this field is outside the Lucene base, but as I
> have hundred of bases like that, with millions of documents for each,
> no I can't change this & reindex the stuff ...
>
> Any other idea ?
Alain,
Since nobody else said it, after a long while...
Your zipped xml could be opened before indexing. You should just index
the data from the xml which will be needed for display. Hopefully that
will consume less index space than the whole xml.
cheers -- Rick
Re: Fetch a binary field
Posted by Barbet Alain <al...@gmail.com>.
Hi !
Because this field containt a zipped xml that is bigger than all
others fields & I don't need it for searching, just for display. Yes
it would be better if this field is outside the Lucene base, but as I
have hundred of bases like that, with millions of documents for each,
no I can't change this & reindex the stuff ...
Any other idea ?
2017-08-11 23:55 GMT+02:00 Dave <ha...@gmail.com>:
> Why didn't you set it to be indexed? Sure it would be a small dent in an index
>
>> On Aug 11, 2017, at 5:20 PM, Barbet Alain <al...@gmail.com> wrote:
>>
>> Re,
>> I take a look on the source code where this msg happen
>> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/schema/SchemaField.java#L186
>>
>> I use version 6.5 who differ from master.
>> In 6.5:
>> if (! (indexed() || hasDocValues()) ) {
>> As my field is not indexed (and hasDocValues() is false as binary),
>> this test fail.
>> Ok but so, what is the way to get this field as I can see it in Luke
>> (in hex format) ?
>>
>>
>> 2017-08-11 15:41 GMT+02:00 Barbet Alain <al...@gmail.com>:
>>> Hi !
>>>
>>>
>>> I've a Lucene base coming from a C++ program linked with Lucene++, a
>>> port of Lucene 3.5.9. When I open this base with Luke, it show Lucene
>>> 2.9. Can see a binary field I have in Luke, with data encoded in
>>> base64.
>>>
>>>
>>> I have upgrade this base from 2.9 => 4.0 => 5.0 =>6.0 so I can use it
>>> with Solr 6.5.1. I rebuild a schema for this base, who have one field
>>> in binary:
>>>
>>>
>>> <field name="document" type="binary" indexed="false" stored="true"/>
>>>
>>>
>>> When I try to retrieve this field with a query (with SOLR admin
>>> interface) it's fail with "can not use FieldCache on a field which is
>>> neither indexed nor has doc values: document"
>>>
>>>
>>> I can retrieve others fields, but can't find a way for this one. Does
>>> someone have an idea ? (At the end I want do this with php, but as it
>>> fail with Solr interface too ....).
>>>
>>> Thank you for any help !
Re: Fetch a binary field
Posted by Dave <ha...@gmail.com>.
Why didn't you set it to be indexed? Sure it would be a small dent in an index
> On Aug 11, 2017, at 5:20 PM, Barbet Alain <al...@gmail.com> wrote:
>
> Re,
> I take a look on the source code where this msg happen
> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/schema/SchemaField.java#L186
>
> I use version 6.5 who differ from master.
> In 6.5:
> if (! (indexed() || hasDocValues()) ) {
> As my field is not indexed (and hasDocValues() is false as binary),
> this test fail.
> Ok but so, what is the way to get this field as I can see it in Luke
> (in hex format) ?
>
>
> 2017-08-11 15:41 GMT+02:00 Barbet Alain <al...@gmail.com>:
>> Hi !
>>
>>
>> I've a Lucene base coming from a C++ program linked with Lucene++, a
>> port of Lucene 3.5.9. When I open this base with Luke, it show Lucene
>> 2.9. Can see a binary field I have in Luke, with data encoded in
>> base64.
>>
>>
>> I have upgrade this base from 2.9 => 4.0 => 5.0 =>6.0 so I can use it
>> with Solr 6.5.1. I rebuild a schema for this base, who have one field
>> in binary:
>>
>>
>> <field name="document" type="binary" indexed="false" stored="true"/>
>>
>>
>> When I try to retrieve this field with a query (with SOLR admin
>> interface) it's fail with "can not use FieldCache on a field which is
>> neither indexed nor has doc values: document"
>>
>>
>> I can retrieve others fields, but can't find a way for this one. Does
>> someone have an idea ? (At the end I want do this with php, but as it
>> fail with Solr interface too ....).
>>
>> Thank you for any help !
Re: Fetch a binary field
Posted by Barbet Alain <al...@gmail.com>.
Re,
I take a look on the source code where this msg happen
https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/schema/SchemaField.java#L186
I use version 6.5 who differ from master.
In 6.5:
if (! (indexed() || hasDocValues()) ) {
As my field is not indexed (and hasDocValues() is false as binary),
this test fail.
Ok but so, what is the way to get this field as I can see it in Luke
(in hex format) ?
2017-08-11 15:41 GMT+02:00 Barbet Alain <al...@gmail.com>:
> Hi !
>
>
> I've a Lucene base coming from a C++ program linked with Lucene++, a
> port of Lucene 3.5.9. When I open this base with Luke, it show Lucene
> 2.9. Can see a binary field I have in Luke, with data encoded in
> base64.
>
>
> I have upgrade this base from 2.9 => 4.0 => 5.0 =>6.0 so I can use it
> with Solr 6.5.1. I rebuild a schema for this base, who have one field
> in binary:
>
>
> <field name="document" type="binary" indexed="false" stored="true"/>
>
>
> When I try to retrieve this field with a query (with SOLR admin
> interface) it's fail with "can not use FieldCache on a field which is
> neither indexed nor has doc values: document"
>
>
> I can retrieve others fields, but can't find a way for this one. Does
> someone have an idea ? (At the end I want do this with php, but as it
> fail with Solr interface too ....).
>
> Thank you for any help !