You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Vjeran Marcinko <vm...@gmail.com> on 2017/08/31 17:29:28 UTC
How to load all document fields, together with facet fields?
I zeroed in the problem with my updating documents having facet
fields... What I need is a way to load document with all fields that
existing when I was saving the document, meaning, together with facet
fields.
Anyway, here's the example.
When I add my document to index, my document is having 3 fields:
"name", "category" and 3rd field is SortedSetDocValueFacetField named
also "category". This is the list of fields printed:
name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
category = cars (stored,indexed,tokenized)
dummy = null (indexed,tokenized)
And before adding to index I build the final document via
facetConfig.build(doc) which produces following fields:
facet_category = null (docValuesType=SORTED_SET)
facet_category = category cars (indexed,omitNorms,indexOptions=DOCS)
facet_category = category (indexed,omitNorms,indexOptions=DOCS)
name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
category = cars (stored,indexed,tokenized)
But, when I load the document plainly using IndexSearcher, I get following:
name = firstDoc (stored,indexed,tokenized,omitNorms,indexOptions=DOCS)
category = cars (stored,indexed,tokenized)
Which is not good if I want to update the document, because there are
no facet fields anymore here, so when I update it, these fields get
lost, so my faceted searching afterwards don't work :(
Any suggestion?
-Vjeran
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: How to load all document fields, together with facet fields?
Posted by Michael McCandless <lu...@mikemccandless.com>.
You'll just have to add additional StoredField instances for all those
facet fields as well.
The FacetField is consumed as an inverted field and not directly stored,
though you could do some work and reconstruct it from the binary doc values
that the facet store.
Mike McCandless
http://blog.mikemccandless.com
On Sat, Sep 2, 2017 at 10:08 AM, Vjeran Marcinko <vm...@gmail.com>
wrote:
> Yes, when I load the doc plainly using IndexSearcher, I got the doc,
> but without special faceted fields::
>
> name = firstDoc (stored,indexed,tokenized,omitNorms,indexOptions=DOCS)
> category = cars (stored,indexed,tokenized)
>
> But I need all those faceted fields somehow, such as when I was saving
> the doc first time:
>
> facet_category = null (docValuesType=SORTED_SET)
> facet_category = category cars (indexed,omitNorms,indexOptions=DOCS)
> facet_category = category (indexed,omitNorms,indexOptions=DOCS)
> name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
> category = cars (stored,indexed,tokenized)
>
> So I'm spekaing how to get those "facet_category" fields, not
> "category"... And I need those because when I update the docs, faceted
> search/coutnting doesn't work without those...
>
> -Vjeran
>
>
> On Sat, Sep 2, 2017 at 11:44 AM, Michael McCandless
> <lu...@mikemccandless.com> wrote:
> > Right, you just need another line like this:
> >
> > doc.add(new StoredField("storedcategory", category));
> >
> > Though, since your TextField category is marked as Field.Store.YES, it
> > should have been in the document when you loaded it at search time. Are
> you
> > sure it's not there?
> >
> > Mike McCandless
> >
> > http://blog.mikemccandless.com
> >
> > On Fri, Sep 1, 2017 at 11:44 AM, Vjeran Marcinko <vm...@gmail.com>
> > wrote:
> >>
> >> Hmmm, dunno what you mean... I currently store my simple document like
> >> this:
> >>
> >> doc.add(new StringField("name", name, Field.Store.YES));
> >> doc.add(new TextField("category", category, Field.Store.YES));
> >> doc.add(new SortedSetDocValuesFacetField("category", category));
> >> Document finalDoc = facetConfig.build(doc);
> >>
> >> So you see, "category" is faceted field. And as I said, I can do
> >> faceted search due to this SortedSetDocValuesFacetField, but problem
> >> is when I load the document and then update it in index, then this
> >> faceted fields dissapear, because they were not loaded in plain way.
> >>
> >> -Vjeran
> >>
> >> On Fri, Sep 1, 2017 at 3:02 PM, Michael McCandless
> >> <lu...@mikemccandless.com> wrote:
> >> > You should separately add those fields to your document, using
> >> > StoredField,
> >> > if you want to retrieve their values at search time.
> >> >
> >> > Mike McCandless
> >> >
> >> > http://blog.mikemccandless.com
> >> >
> >> > On Thu, Aug 31, 2017 at 1:29 PM, Vjeran Marcinko <vmarcinko@gmail.com
> >
> >> > wrote:
> >> >>
> >> >> I zeroed in the problem with my updating documents having facet
> >> >> fields... What I need is a way to load document with all fields that
> >> >> existing when I was saving the document, meaning, together with facet
> >> >> fields.
> >> >>
> >> >> Anyway, here's the example.
> >> >>
> >> >> When I add my document to index, my document is having 3 fields:
> >> >> "name", "category" and 3rd field is SortedSetDocValueFacetField named
> >> >> also "category". This is the list of fields printed:
> >> >>
> >> >> name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
> >> >> category = cars (stored,indexed,tokenized)
> >> >> dummy = null (indexed,tokenized)
> >> >>
> >> >> And before adding to index I build the final document via
> >> >> facetConfig.build(doc) which produces following fields:
> >> >>
> >> >> facet_category = null (docValuesType=SORTED_SET)
> >> >> facet_category = category cars (indexed,omitNorms,indexOptions=DOCS)
> >> >> facet_category = category (indexed,omitNorms,indexOptions=DOCS)
> >> >> name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
> >> >> category = cars (stored,indexed,tokenized)
> >> >>
> >> >> But, when I load the document plainly using IndexSearcher, I get
> >> >> following:
> >> >>
> >> >> name = firstDoc (stored,indexed,tokenized,
> omitNorms,indexOptions=DOCS)
> >> >> category = cars (stored,indexed,tokenized)
> >> >>
> >> >> Which is not good if I want to update the document, because there are
> >> >> no facet fields anymore here, so when I update it, these fields get
> >> >> lost, so my faceted searching afterwards don't work :(
> >> >>
> >> >> Any suggestion?
> >> >>
> >> >> -Vjeran
> >> >>
> >> >> ------------------------------------------------------------
> ---------
> >> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >>
> >> >
> >
> >
>
Re: How to load all document fields, together with facet fields?
Posted by Vjeran Marcinko <vm...@gmail.com>.
Yes, when I load the doc plainly using IndexSearcher, I got the doc,
but without special faceted fields::
name = firstDoc (stored,indexed,tokenized,omitNorms,indexOptions=DOCS)
category = cars (stored,indexed,tokenized)
But I need all those faceted fields somehow, such as when I was saving
the doc first time:
facet_category = null (docValuesType=SORTED_SET)
facet_category = category cars (indexed,omitNorms,indexOptions=DOCS)
facet_category = category (indexed,omitNorms,indexOptions=DOCS)
name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
category = cars (stored,indexed,tokenized)
So I'm spekaing how to get those "facet_category" fields, not
"category"... And I need those because when I update the docs, faceted
search/coutnting doesn't work without those...
-Vjeran
On Sat, Sep 2, 2017 at 11:44 AM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> Right, you just need another line like this:
>
> doc.add(new StoredField("storedcategory", category));
>
> Though, since your TextField category is marked as Field.Store.YES, it
> should have been in the document when you loaded it at search time. Are you
> sure it's not there?
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Fri, Sep 1, 2017 at 11:44 AM, Vjeran Marcinko <vm...@gmail.com>
> wrote:
>>
>> Hmmm, dunno what you mean... I currently store my simple document like
>> this:
>>
>> doc.add(new StringField("name", name, Field.Store.YES));
>> doc.add(new TextField("category", category, Field.Store.YES));
>> doc.add(new SortedSetDocValuesFacetField("category", category));
>> Document finalDoc = facetConfig.build(doc);
>>
>> So you see, "category" is faceted field. And as I said, I can do
>> faceted search due to this SortedSetDocValuesFacetField, but problem
>> is when I load the document and then update it in index, then this
>> faceted fields dissapear, because they were not loaded in plain way.
>>
>> -Vjeran
>>
>> On Fri, Sep 1, 2017 at 3:02 PM, Michael McCandless
>> <lu...@mikemccandless.com> wrote:
>> > You should separately add those fields to your document, using
>> > StoredField,
>> > if you want to retrieve their values at search time.
>> >
>> > Mike McCandless
>> >
>> > http://blog.mikemccandless.com
>> >
>> > On Thu, Aug 31, 2017 at 1:29 PM, Vjeran Marcinko <vm...@gmail.com>
>> > wrote:
>> >>
>> >> I zeroed in the problem with my updating documents having facet
>> >> fields... What I need is a way to load document with all fields that
>> >> existing when I was saving the document, meaning, together with facet
>> >> fields.
>> >>
>> >> Anyway, here's the example.
>> >>
>> >> When I add my document to index, my document is having 3 fields:
>> >> "name", "category" and 3rd field is SortedSetDocValueFacetField named
>> >> also "category". This is the list of fields printed:
>> >>
>> >> name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
>> >> category = cars (stored,indexed,tokenized)
>> >> dummy = null (indexed,tokenized)
>> >>
>> >> And before adding to index I build the final document via
>> >> facetConfig.build(doc) which produces following fields:
>> >>
>> >> facet_category = null (docValuesType=SORTED_SET)
>> >> facet_category = category cars (indexed,omitNorms,indexOptions=DOCS)
>> >> facet_category = category (indexed,omitNorms,indexOptions=DOCS)
>> >> name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
>> >> category = cars (stored,indexed,tokenized)
>> >>
>> >> But, when I load the document plainly using IndexSearcher, I get
>> >> following:
>> >>
>> >> name = firstDoc (stored,indexed,tokenized,omitNorms,indexOptions=DOCS)
>> >> category = cars (stored,indexed,tokenized)
>> >>
>> >> Which is not good if I want to update the document, because there are
>> >> no facet fields anymore here, so when I update it, these fields get
>> >> lost, so my faceted searching afterwards don't work :(
>> >>
>> >> Any suggestion?
>> >>
>> >> -Vjeran
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: java-user-help@lucene.apache.org
>> >>
>> >
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: How to load all document fields, together with facet fields?
Posted by Michael McCandless <lu...@mikemccandless.com>.
Right, you just need another line like this:
doc.add(new StoredField("storedcategory", category));
Though, since your TextField category is marked as Field.Store.YES, it
should have been in the document when you loaded it at search time. Are
you sure it's not there?
Mike McCandless
http://blog.mikemccandless.com
On Fri, Sep 1, 2017 at 11:44 AM, Vjeran Marcinko <vm...@gmail.com>
wrote:
> Hmmm, dunno what you mean... I currently store my simple document like
> this:
>
> doc.add(new StringField("name", name, Field.Store.YES));
> doc.add(new TextField("category", category, Field.Store.YES));
> doc.add(new SortedSetDocValuesFacetField("category", category));
> Document finalDoc = facetConfig.build(doc);
>
> So you see, "category" is faceted field. And as I said, I can do
> faceted search due to this SortedSetDocValuesFacetField, but problem
> is when I load the document and then update it in index, then this
> faceted fields dissapear, because they were not loaded in plain way.
>
> -Vjeran
>
> On Fri, Sep 1, 2017 at 3:02 PM, Michael McCandless
> <lu...@mikemccandless.com> wrote:
> > You should separately add those fields to your document, using
> StoredField,
> > if you want to retrieve their values at search time.
> >
> > Mike McCandless
> >
> > http://blog.mikemccandless.com
> >
> > On Thu, Aug 31, 2017 at 1:29 PM, Vjeran Marcinko <vm...@gmail.com>
> > wrote:
> >>
> >> I zeroed in the problem with my updating documents having facet
> >> fields... What I need is a way to load document with all fields that
> >> existing when I was saving the document, meaning, together with facet
> >> fields.
> >>
> >> Anyway, here's the example.
> >>
> >> When I add my document to index, my document is having 3 fields:
> >> "name", "category" and 3rd field is SortedSetDocValueFacetField named
> >> also "category". This is the list of fields printed:
> >>
> >> name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
> >> category = cars (stored,indexed,tokenized)
> >> dummy = null (indexed,tokenized)
> >>
> >> And before adding to index I build the final document via
> >> facetConfig.build(doc) which produces following fields:
> >>
> >> facet_category = null (docValuesType=SORTED_SET)
> >> facet_category = category cars (indexed,omitNorms,indexOptions=DOCS)
> >> facet_category = category (indexed,omitNorms,indexOptions=DOCS)
> >> name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
> >> category = cars (stored,indexed,tokenized)
> >>
> >> But, when I load the document plainly using IndexSearcher, I get
> >> following:
> >>
> >> name = firstDoc (stored,indexed,tokenized,omitNorms,indexOptions=DOCS)
> >> category = cars (stored,indexed,tokenized)
> >>
> >> Which is not good if I want to update the document, because there are
> >> no facet fields anymore here, so when I update it, these fields get
> >> lost, so my faceted searching afterwards don't work :(
> >>
> >> Any suggestion?
> >>
> >> -Vjeran
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >
>
Re: How to load all document fields, together with facet fields?
Posted by Vjeran Marcinko <vm...@gmail.com>.
Hmmm, dunno what you mean... I currently store my simple document like this:
doc.add(new StringField("name", name, Field.Store.YES));
doc.add(new TextField("category", category, Field.Store.YES));
doc.add(new SortedSetDocValuesFacetField("category", category));
Document finalDoc = facetConfig.build(doc);
So you see, "category" is faceted field. And as I said, I can do
faceted search due to this SortedSetDocValuesFacetField, but problem
is when I load the document and then update it in index, then this
faceted fields dissapear, because they were not loaded in plain way.
-Vjeran
On Fri, Sep 1, 2017 at 3:02 PM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> You should separately add those fields to your document, using StoredField,
> if you want to retrieve their values at search time.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Thu, Aug 31, 2017 at 1:29 PM, Vjeran Marcinko <vm...@gmail.com>
> wrote:
>>
>> I zeroed in the problem with my updating documents having facet
>> fields... What I need is a way to load document with all fields that
>> existing when I was saving the document, meaning, together with facet
>> fields.
>>
>> Anyway, here's the example.
>>
>> When I add my document to index, my document is having 3 fields:
>> "name", "category" and 3rd field is SortedSetDocValueFacetField named
>> also "category". This is the list of fields printed:
>>
>> name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
>> category = cars (stored,indexed,tokenized)
>> dummy = null (indexed,tokenized)
>>
>> And before adding to index I build the final document via
>> facetConfig.build(doc) which produces following fields:
>>
>> facet_category = null (docValuesType=SORTED_SET)
>> facet_category = category cars (indexed,omitNorms,indexOptions=DOCS)
>> facet_category = category (indexed,omitNorms,indexOptions=DOCS)
>> name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
>> category = cars (stored,indexed,tokenized)
>>
>> But, when I load the document plainly using IndexSearcher, I get
>> following:
>>
>> name = firstDoc (stored,indexed,tokenized,omitNorms,indexOptions=DOCS)
>> category = cars (stored,indexed,tokenized)
>>
>> Which is not good if I want to update the document, because there are
>> no facet fields anymore here, so when I update it, these fields get
>> lost, so my faceted searching afterwards don't work :(
>>
>> Any suggestion?
>>
>> -Vjeran
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: How to load all document fields, together with facet fields?
Posted by Michael McCandless <lu...@mikemccandless.com>.
You should separately add those fields to your document, using StoredField,
if you want to retrieve their values at search time.
Mike McCandless
http://blog.mikemccandless.com
On Thu, Aug 31, 2017 at 1:29 PM, Vjeran Marcinko <vm...@gmail.com>
wrote:
> I zeroed in the problem with my updating documents having facet
> fields... What I need is a way to load document with all fields that
> existing when I was saving the document, meaning, together with facet
> fields.
>
> Anyway, here's the example.
>
> When I add my document to index, my document is having 3 fields:
> "name", "category" and 3rd field is SortedSetDocValueFacetField named
> also "category". This is the list of fields printed:
>
> name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
> category = cars (stored,indexed,tokenized)
> dummy = null (indexed,tokenized)
>
> And before adding to index I build the final document via
> facetConfig.build(doc) which produces following fields:
>
> facet_category = null (docValuesType=SORTED_SET)
> facet_category = category cars (indexed,omitNorms,indexOptions=DOCS)
> facet_category = category (indexed,omitNorms,indexOptions=DOCS)
> name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
> category = cars (stored,indexed,tokenized)
>
> But, when I load the document plainly using IndexSearcher, I get following:
>
> name = firstDoc (stored,indexed,tokenized,omitNorms,indexOptions=DOCS)
> category = cars (stored,indexed,tokenized)
>
> Which is not good if I want to update the document, because there are
> no facet fields anymore here, so when I update it, these fields get
> lost, so my faceted searching afterwards don't work :(
>
> Any suggestion?
>
> -Vjeran
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: How to load all document fields, together with facet fields?
Posted by kribsky <kr...@atlascon.cz>.
Hi!
if you want to read facetized fields you need to search trough facet
collector. For example like this
FacetsCollector facetsCollector =new FacetsCollector();
FacetsCollector.search(indexSearcher, query, pageSize, facetsCollector);
FastTaxonomyFacetCounts customFastFacetCounts =new FastTaxonomyFacetCounts(taxonomyReader,config, facetsCollector);
allDims = customFastFacetCounts.getAllDims(topDims);
Where all dims is facitized fields. I hope this helps you.
Kribsky Tomas
On 31.8.2017 19:29, Vjeran Marcinko wrote:
> I zeroed in the problem with my updating documents having facet
> fields... What I need is a way to load document with all fields that
> existing when I was saving the document, meaning, together with facet
> fields.
>
> Anyway, here's the example.
>
> When I add my document to index, my document is having 3 fields:
> "name", "category" and 3rd field is SortedSetDocValueFacetField named
> also "category". This is the list of fields printed:
>
> name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
> category = cars (stored,indexed,tokenized)
> dummy = null (indexed,tokenized)
>
> And before adding to index I build the final document via
> facetConfig.build(doc) which produces following fields:
>
> facet_category = null (docValuesType=SORTED_SET)
> facet_category = category cars (indexed,omitNorms,indexOptions=DOCS)
> facet_category = category (indexed,omitNorms,indexOptions=DOCS)
> name = firstDoc (stored,indexed,omitNorms,indexOptions=DOCS)
> category = cars (stored,indexed,tokenized)
>
> But, when I load the document plainly using IndexSearcher, I get following:
>
> name = firstDoc (stored,indexed,tokenized,omitNorms,indexOptions=DOCS)
> category = cars (stored,indexed,tokenized)
>
> Which is not good if I want to update the document, because there are
> no facet fields anymore here, so when I update it, these fields get
> lost, so my faceted searching afterwards don't work :(
>
> Any suggestion?
>
> -Vjeran
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>