You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Anil <an...@gmail.com> on 2016/02/24 12:53:32 UTC
WhitespaceTokenizerFactory and PathHierarchyTokenizerFactory
HI,
i need to use both WhitespaceTokenizerFactory and
PathHierarchyTokenizerFactory for use case.
Solr supports only one tokenizer. is there any way we can achieve
PathHierarchyTokenizerFactory functionality with filters ?
Please advice.
Regards,
Anil
Re: WhitespaceTokenizerFactory and PathHierarchyTokenizerFactory
Posted by Anil <an...@gmail.com>.
HI,
search can be any free text or ip address or path and Special characters
should not be treated as text delimiters.
10.20 must return 10.20.30.112
/var/log must return /var/log/bigdata
Please let me know if you need any additional details. Thanks.
Regards,
Anil
On 25 February 2016 at 18:20, Jack Krupansky <ja...@gmail.com>
wrote:
> You still haven't stated exactly what your query requirements are. In Solr
> you should always start with an analysis of how people will expect to query
> the data and then work backwards to how to store and index the data to
> achieve the desired queries.
>
> Note that the standard tokenizer will tokenize all of the elements of a
> path or IP as separate terms. Ditto for a query, so you can effectively do
> bth keyword and phrase queries to match individual terms (e.g., path
> elements) or phrases or sequences of path elements or IP address
> components.
>
> -- Jack Krupansky
>
> On Thu, Feb 25, 2016 at 12:41 AM, Anil <an...@gmail.com> wrote:
>
> > Sorry Jack for confusion.
> >
> > I have field which holds free text. text can contain path , ip or any
> free
> > text.
> >
> > I would like to tokenize the text of the field using white space. if the
> > text token is of path or ip pattern , it has be tockenized like path
> > hierarchy way.
> >
> >
> > Regards,
> > Anil
> >
> > On 24 February 2016 at 21:59, Jack Krupansky <ja...@gmail.com>
> > wrote:
> >
> > > Your statement makes no sense. Please clarify. Express your
> > requirement(s)
> > > in plain English first before dragging in possible solutions.
> > Technically,
> > > path elements can have embedded spaces.
> > >
> > > -- Jack Krupansky
> > >
> > > On Wed, Feb 24, 2016 at 6:53 AM, Anil <an...@gmail.com> wrote:
> > >
> > > > HI,
> > > >
> > > > i need to use both WhitespaceTokenizerFactory and
> > > > PathHierarchyTokenizerFactory for use case.
> > > >
> > > > Solr supports only one tokenizer. is there any way we can achieve
> > > > PathHierarchyTokenizerFactory functionality with filters ?
> > > >
> > > > Please advice.
> > > >
> > > > Regards,
> > > > Anil
> > > >
> > >
> >
>
Re: WhitespaceTokenizerFactory and PathHierarchyTokenizerFactory
Posted by Jack Krupansky <ja...@gmail.com>.
You still haven't stated exactly what your query requirements are. In Solr
you should always start with an analysis of how people will expect to query
the data and then work backwards to how to store and index the data to
achieve the desired queries.
Note that the standard tokenizer will tokenize all of the elements of a
path or IP as separate terms. Ditto for a query, so you can effectively do
bth keyword and phrase queries to match individual terms (e.g., path
elements) or phrases or sequences of path elements or IP address components.
-- Jack Krupansky
On Thu, Feb 25, 2016 at 12:41 AM, Anil <an...@gmail.com> wrote:
> Sorry Jack for confusion.
>
> I have field which holds free text. text can contain path , ip or any free
> text.
>
> I would like to tokenize the text of the field using white space. if the
> text token is of path or ip pattern , it has be tockenized like path
> hierarchy way.
>
>
> Regards,
> Anil
>
> On 24 February 2016 at 21:59, Jack Krupansky <ja...@gmail.com>
> wrote:
>
> > Your statement makes no sense. Please clarify. Express your
> requirement(s)
> > in plain English first before dragging in possible solutions.
> Technically,
> > path elements can have embedded spaces.
> >
> > -- Jack Krupansky
> >
> > On Wed, Feb 24, 2016 at 6:53 AM, Anil <an...@gmail.com> wrote:
> >
> > > HI,
> > >
> > > i need to use both WhitespaceTokenizerFactory and
> > > PathHierarchyTokenizerFactory for use case.
> > >
> > > Solr supports only one tokenizer. is there any way we can achieve
> > > PathHierarchyTokenizerFactory functionality with filters ?
> > >
> > > Please advice.
> > >
> > > Regards,
> > > Anil
> > >
> >
>
Re: WhitespaceTokenizerFactory and PathHierarchyTokenizerFactory
Posted by Anil <an...@gmail.com>.
Sorry Jack for confusion.
I have field which holds free text. text can contain path , ip or any free
text.
I would like to tokenize the text of the field using white space. if the
text token is of path or ip pattern , it has be tockenized like path
hierarchy way.
Regards,
Anil
On 24 February 2016 at 21:59, Jack Krupansky <ja...@gmail.com>
wrote:
> Your statement makes no sense. Please clarify. Express your requirement(s)
> in plain English first before dragging in possible solutions. Technically,
> path elements can have embedded spaces.
>
> -- Jack Krupansky
>
> On Wed, Feb 24, 2016 at 6:53 AM, Anil <an...@gmail.com> wrote:
>
> > HI,
> >
> > i need to use both WhitespaceTokenizerFactory and
> > PathHierarchyTokenizerFactory for use case.
> >
> > Solr supports only one tokenizer. is there any way we can achieve
> > PathHierarchyTokenizerFactory functionality with filters ?
> >
> > Please advice.
> >
> > Regards,
> > Anil
> >
>
Re: WhitespaceTokenizerFactory and PathHierarchyTokenizerFactory
Posted by Jack Krupansky <ja...@gmail.com>.
Your statement makes no sense. Please clarify. Express your requirement(s)
in plain English first before dragging in possible solutions. Technically,
path elements can have embedded spaces.
-- Jack Krupansky
On Wed, Feb 24, 2016 at 6:53 AM, Anil <an...@gmail.com> wrote:
> HI,
>
> i need to use both WhitespaceTokenizerFactory and
> PathHierarchyTokenizerFactory for use case.
>
> Solr supports only one tokenizer. is there any way we can achieve
> PathHierarchyTokenizerFactory functionality with filters ?
>
> Please advice.
>
> Regards,
> Anil
>