Not seeing the tokenized values when using solr.PathHierarchyTokenizerFactory

[Mark Robinson <ma...@gmail.com>]

Hi,

I was using the solr.PathHierarchyTokenizerFactory for a field say fieldB.
An input data like A/B/C when I check using the ANALYSIS facility in the
admin UI, is tokenized as A, A/B, A/B/C in fieldB.
A/B/C in my system is a "string" value in a fieldA which is both
indexed=stored=true. I copyField fieldA to fieldB which has the above
solr.PathHierarchyTokenizerFactory.

fieldB also has indexed=stored=true as well as multiValued=true.

Even then, when results displayed, fieldB shows only the original A/B/C ie
same as what is in fieldA.
But ANALYSIS as mentioned above shows all the different hierarchies for
fieldB. Also a querylike:-  fieldA:"A/B" yields no results but
fieldB:"A/B" gives results as fieldB has all the hierarchies in it.

But then why can't I see all the different hierarchies in my result for
fieldB as I clearly see when I check through the ANALYSIS in admin UI?

Could some one pls help understand this behavior.

Thanks!
Mark.

Re: Not seeing the tokenized values when using solr.PathHierarchyTokenizerFactory

[Mark Robinson <ma...@gmail.com>]

Thanks much Eric!
Got it.

Best,
Mark.

On Mon, Apr 18, 2016 at 7:53 PM, Erick Erickson <er...@gmail.com>
wrote:

> Assuming that you're talking about the docs returned in the result
> sets, these are the _stored_ fields, not the analyzed field. Stored
> fields are a verbatim copy of the original input.
>
> Best,
> Erick
>
> On Mon, Apr 18, 2016 at 12:51 PM, Mark Robinson <ma...@gmail.com>
> wrote:
> > Hi,
> >
> > I was using the solr.PathHierarchyTokenizerFactory for a field say
> fieldB.
> > An input data like A/B/C when I check using the ANALYSIS facility in the
> > admin UI, is tokenized as A, A/B, A/B/C in fieldB.
> > A/B/C in my system is a "string" value in a fieldA which is both
> > indexed=stored=true. I copyField fieldA to fieldB which has the above
> > solr.PathHierarchyTokenizerFactory.
> >
> > fieldB also has indexed=stored=true as well as multiValued=true.
> >
> > Even then, when results displayed, fieldB shows only the original A/B/C
> ie
> > same as what is in fieldA.
> > But ANALYSIS as mentioned above shows all the different hierarchies for
> > fieldB. Also a querylike:-  fieldA:"A/B" yields no results but
> > fieldB:"A/B" gives results as fieldB has all the hierarchies in it.
> >
> > But then why can't I see all the different hierarchies in my result for
> > fieldB as I clearly see when I check through the ANALYSIS in admin UI?
> >
> > Could some one pls help understand this behavior.
> >
> > Thanks!
> > Mark.
>

Re: Not seeing the tokenized values when using solr.PathHierarchyTokenizerFactory

[Erick Erickson <er...@gmail.com>]

Assuming that you're talking about the docs returned in the result
sets, these are the _stored_ fields, not the analyzed field. Stored
fields are a verbatim copy of the original input.

Best,
Erick

On Mon, Apr 18, 2016 at 12:51 PM, Mark Robinson <ma...@gmail.com> wrote:
> Hi,
>
> I was using the solr.PathHierarchyTokenizerFactory for a field say fieldB.
> An input data like A/B/C when I check using the ANALYSIS facility in the
> admin UI, is tokenized as A, A/B, A/B/C in fieldB.
> A/B/C in my system is a "string" value in a fieldA which is both
> indexed=stored=true. I copyField fieldA to fieldB which has the above
> solr.PathHierarchyTokenizerFactory.
>
> fieldB also has indexed=stored=true as well as multiValued=true.
>
> Even then, when results displayed, fieldB shows only the original A/B/C ie
> same as what is in fieldA.
> But ANALYSIS as mentioned above shows all the different hierarchies for
> fieldB. Also a querylike:-  fieldA:"A/B" yields no results but
> fieldB:"A/B" gives results as fieldB has all the hierarchies in it.
>
> But then why can't I see all the different hierarchies in my result for
> fieldB as I clearly see when I check through the ANALYSIS in admin UI?
>
> Could some one pls help understand this behavior.
>
> Thanks!
> Mark.