You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@struts.apache.org by Robert Taylor <rt...@mulework.com> on 2004/03/15 22:20:11 UTC

[OT] Search string tokenizer

I did a google search on this and didn't really come up with anything useful.
Before I implement this myself, is there an existing implementation of parsing
a search string which would produce tokens similar to how Google or other search
engines parse search strings.

For example, I would like to parse a search string into tokens where tokens are 
delimited by either a blank space or a quoted phrase.

So the string:

'Struts "web presentation tier"' 

would return  2 tokens:
 - Struts
 - web presentation tier

but the string:

'Struts web presentation tier'

would return 4 tokens:
 - Struts
 - web
 - presentation
 - tier


Any help is appreciated.

robert







---------------------------------------------------------------------
To unsubscribe, e-mail: struts-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: struts-user-help@jakarta.apache.org


Re: [OT] Search string tokenizer

Posted by Michael McGrady <mi...@michaelmcgrady.com>.
I have not investigated this at all, but Lucene should, if it doesn't, do this.

At 01:20 PM 3/15/2004, you wrote:
>I did a google search on this and didn't really come up with anything useful.
>Before I implement this myself, is there an existing implementation of parsing
>a search string which would produce tokens similar to how Google or other 
>search
>engines parse search strings.
>
>For example, I would like to parse a search string into tokens where 
>tokens are
>delimited by either a blank space or a quoted phrase.
>
>So the string:
>
>'Struts "web presentation tier"'
>
>would return  2 tokens:
>  - Struts
>  - web presentation tier
>
>but the string:
>
>'Struts web presentation tier'
>
>would return 4 tokens:
>  - Struts
>  - web
>  - presentation
>  - tier
>
>
>Any help is appreciated.
>
>robert
>
>
>
>
>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: struts-user-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: struts-user-help@jakarta.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: struts-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: struts-user-help@jakarta.apache.org


RE: [OT] Search string tokenizer

Posted by Robert Taylor <rt...@mulework.com>.
Jason and Larry, I think I'm going to investigate using regexp.

Michael, I looked into Lucene and I really don't need something that "heavy".
I just need to perform some simple searches against the database.

Thanks to everyone who responded on and off list.

robert



> -----Original Message-----
> From: Jason Lea [mailto:jason@kumachan.net.nz]
> Sent: Monday, March 15, 2004 5:30 PM
> To: Struts Users Mailing List
> Subject: Re: [OT] Search string tokenizer
> 
> 
> Regular Expressions should work:
> 
> There are a few implementations too like 
> http://jakarta.apache.org/oro/index.html
> 
> With a pattern like this:  (".+"|[\d\w]*)
> You should get close to what you need (i think the above will return the 
> space character as a token too, not sure).
> 
> And here is a program that lets you test out patterns to see how they 
> match strings or will be split
> http://www.weitz.de/regex-coach/
> 
> Robert Taylor wrote:
> 
> >I did a google search on this and didn't really come up with anything useful.
> >Before I implement this myself, is there an existing implementation of parsing
> >a search string which would produce tokens similar to how Google or other search
> >engines parse search strings.
> >
> >For example, I would like to parse a search string into tokens where tokens are 
> >delimited by either a blank space or a quoted phrase.
> >
> >So the string:
> >
> >'Struts "web presentation tier"' 
> >
> >would return  2 tokens:
> > - Struts
> > - web presentation tier
> >
> >but the string:
> >
> >'Struts web presentation tier'
> >
> >would return 4 tokens:
> > - Struts
> > - web
> > - presentation
> > - tier
> >
> >
> >Any help is appreciated.
> >
> >robert
> >
> >
> >
> >
> >
> >
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: struts-user-unsubscribe@jakarta.apache.org
> >For additional commands, e-mail: struts-user-help@jakarta.apache.org
> >
> >
> >  
> >
> 
> 
> -- 
> Jason Lea
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: struts-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: struts-user-help@jakarta.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: struts-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: struts-user-help@jakarta.apache.org


Re: [OT] Search string tokenizer

Posted by Jason Lea <ja...@kumachan.net.nz>.
Regular Expressions should work:

There are a few implementations too like 
http://jakarta.apache.org/oro/index.html

With a pattern like this:  (".+"|[\d\w]*)
You should get close to what you need (i think the above will return the 
space character as a token too, not sure).

And here is a program that lets you test out patterns to see how they 
match strings or will be split
http://www.weitz.de/regex-coach/

Robert Taylor wrote:

>I did a google search on this and didn't really come up with anything useful.
>Before I implement this myself, is there an existing implementation of parsing
>a search string which would produce tokens similar to how Google or other search
>engines parse search strings.
>
>For example, I would like to parse a search string into tokens where tokens are 
>delimited by either a blank space or a quoted phrase.
>
>So the string:
>
>'Struts "web presentation tier"' 
>
>would return  2 tokens:
> - Struts
> - web presentation tier
>
>but the string:
>
>'Struts web presentation tier'
>
>would return 4 tokens:
> - Struts
> - web
> - presentation
> - tier
>
>
>Any help is appreciated.
>
>robert
>
>
>
>
>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: struts-user-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: struts-user-help@jakarta.apache.org
>
>
>  
>


-- 
Jason Lea



---------------------------------------------------------------------
To unsubscribe, e-mail: struts-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: struts-user-help@jakarta.apache.org