You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Osma Suominen <os...@helsinki.fi> on 2014/09/01 14:38:25 UTC

jena-text LowerCaseKeywordAnalyzer implementation

Hi!

I liked the option to specify Analyzer for jena-text, as implemented in 
JENA-654. But I'd like to use an analyzer that is otherwise like 
KeywordAnalyzer but case-insensitive, for use in an 
autocomplete/typeahead UI widget. Lucene doesn't include such an 
analyzer, but there are several implementations of the same idea, e.g. 
in neo4j [1] and stargate [2].

I created my own implementation of such an analyzer and added code to 
use it from the assembler. Patch attached.

This analyzer is now in a new package 
org.apache.jena.query.text.analyzer, in case other analyzers for 
jena-text will appear in the future. If you don't like the new package, 
the class can of course be moved to org.apache.jena.query.text.

I also added a test for case-insensitivity. To avoid lots of duplicate 
boilerplate code, I slightly modified and subclassed the existing test 
for KeywordAnalyzer.

I'd love to see this in the next version of jena-text and Fuseki. Of 
course I'll rework the patch if necessary. I can also tweak the web 
documentation to mention this analyzer.

-Osma


[1] 
https://github.com/apatry/neo4j-lucene4-index/blob/master/src/main/java/org/neo4j/index/impl/lucene/LowerCaseKeywordAnalyzer.java

[2] 
https://github.com/tuplejump/stargate-core/blob/master/src/main/java/com/tuplejump/stargate/lucene/CaseInsensitiveKeywordAnalyzer.java

-- 
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Teollisuuskatu 23)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi

Re: jena-text LowerCaseKeywordAnalyzer implementation

Posted by Osma Suominen <os...@helsinki.fi>.
Thanks a lot Andy!

-Osma

On 04/09/14 11:47, Andy Seaborne wrote:
> Hi Osma,
>
> Patch applied and documentation updated - thank you very much.
>
> I kicked off a deployment build so quite soon (~15 mins) there'll be an
> updated snapshot development build with this in.
>
>      Andy
>
> On 04/09/14 07:22, Osma Suominen wrote:
>> Hi Andy!
>>
>> 03.09.2014, 14:32, Andy Seaborne wrote:
>>>> https://issues.apache.org/jira/browse/JENA-776
>>> Thanks.
>>
>> I couldn't find a way to assign this to you though, it's currently still
>> Unassigned. Maybe I don't have permission to do that in Jira.
>>
>>> Just putting the new text in a JIRA comment and a description of where
>>> it goes on the page (just to make sure it's clear), is fine for a minor
>>> change.
>>
>> OK. It's just adding one sentence. I put the description of what to do
>> as a Jira comment.
>>
>> -Osma
>>
>


-- 
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Teollisuuskatu 23)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi

Re: jena-text LowerCaseKeywordAnalyzer implementation

Posted by Andy Seaborne <an...@apache.org>.
Hi Osma,

Patch applied and documentation updated - thank you very much.

I kicked off a deployment build so quite soon (~15 mins) there'll be an 
updated snapshot development build with this in.

	Andy

On 04/09/14 07:22, Osma Suominen wrote:
> Hi Andy!
>
> 03.09.2014, 14:32, Andy Seaborne wrote:
>>> https://issues.apache.org/jira/browse/JENA-776
>> Thanks.
>
> I couldn't find a way to assign this to you though, it's currently still
> Unassigned. Maybe I don't have permission to do that in Jira.
>
>> Just putting the new text in a JIRA comment and a description of where
>> it goes on the page (just to make sure it's clear), is fine for a minor
>> change.
>
> OK. It's just adding one sentence. I put the description of what to do
> as a Jira comment.
>
> -Osma
>


Re: jena-text LowerCaseKeywordAnalyzer implementation

Posted by Osma Suominen <os...@helsinki.fi>.
Hi Andy!

03.09.2014, 14:32, Andy Seaborne wrote:
>> https://issues.apache.org/jira/browse/JENA-776
> Thanks.

I couldn't find a way to assign this to you though, it's currently still 
Unassigned. Maybe I don't have permission to do that in Jira.

> Just putting the new text in a JIRA comment and a description of where
> it goes on the page (just to make sure it's clear), is fine for a minor
> change.

OK. It's just adding one sentence. I put the description of what to do 
as a Jira comment.

-Osma

-- 
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Teollisuuskatu 23)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi

Re: jena-text LowerCaseKeywordAnalyzer implementation

Posted by Andy Seaborne <an...@apache.org>.
On 03/09/14 12:21, Osma Suominen wrote:
> Hi Andy!
>
> Thanks for your response!
>
> 03.09.2014, 13:29, Andy Seaborne wrote:
>
>> What would be helpful would be:
>>
>> 1/ Create JIRA and attach the patch (it's less likely to get lost!)
>>     You can assign it to me.  I have just checked the patch applies OK
>>     for me.
>
> Done:
> https://issues.apache.org/jira/browse/JENA-776

Thanks.

>
>> 2/ Is there any documentation updates to go with this for:
>>
>> http://jena.apache.org/documentation/query/text-query.html
>>
>> whose source is in markdown at:
>>
>> https://svn.apache.org/repos/asf/jena/site/trunk/content/documentation/query/text-query.mdtext
>>
>
> Yes, will update. It's just a minor edit, mentioning the new Analyzer.
> Last time I used the ASF CMS to update the jena-text documentation (that
> was for graph-specific indexing). Is that still the way to do it? Or
> should I create a patch for the Markdown file?

Yes - that should still work.

Just putting the new text in a JIRA comment and a description of where 
it goes on the page (just to make sure it's clear), is fine for a minor 
change.

	Andy

>
> -Osma
>


Re: jena-text LowerCaseKeywordAnalyzer implementation

Posted by Osma Suominen <os...@helsinki.fi>.
Hi Andy!

Thanks for your response!

03.09.2014, 13:29, Andy Seaborne wrote:

> What would be helpful would be:
>
> 1/ Create JIRA and attach the patch (it's less likely to get lost!)
>     You can assign it to me.  I have just checked the patch applies OK
>     for me.

Done:
https://issues.apache.org/jira/browse/JENA-776

> 2/ Is there any documentation updates to go with this for:
>
> http://jena.apache.org/documentation/query/text-query.html
>
> whose source is in markdown at:
>
> https://svn.apache.org/repos/asf/jena/site/trunk/content/documentation/query/text-query.mdtext

Yes, will update. It's just a minor edit, mentioning the new Analyzer. 
Last time I used the ASF CMS to update the jena-text documentation (that 
was for graph-specific indexing). Is that still the way to do it? Or 
should I create a patch for the Markdown file?

-Osma

-- 
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Teollisuuskatu 23)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi

Re: jena-text LowerCaseKeywordAnalyzer implementation

Posted by Andy Seaborne <an...@apache.org>.
On 03/09/14 11:07, Osma Suominen wrote:
> Hi!
>
> Any interest in my contribution? Anything I can do to help get this, or
> something like it, into jena-text?

Yes - it looks interesting.

What would be helpful would be:

1/ Create JIRA and attach the patch (it's less likely to get lost!)
    You can assign it to me.  I have just checked the patch applies OK
    for me.

2/ Is there any documentation updates to go with this for:

http://jena.apache.org/documentation/query/text-query.html

whose source is in markdown at:

https://svn.apache.org/repos/asf/jena/site/trunk/content/documentation/query/text-query.mdtext

	Andy

>
> -Osma
>
> 01.09.2014, 15:38, Osma Suominen kirjoitti:
>> Hi!
>>
>> I liked the option to specify Analyzer for jena-text, as implemented in
>> JENA-654. But I'd like to use an analyzer that is otherwise like
>> KeywordAnalyzer but case-insensitive, for use in an
>> autocomplete/typeahead UI widget. Lucene doesn't include such an
>> analyzer, but there are several implementations of the same idea, e.g.
>> in neo4j [1] and stargate [2].
>>
>> I created my own implementation of such an analyzer and added code to
>> use it from the assembler. Patch attached.
>>
>> This analyzer is now in a new package
>> org.apache.jena.query.text.analyzer, in case other analyzers for
>> jena-text will appear in the future. If you don't like the new package,
>> the class can of course be moved to org.apache.jena.query.text.
>>
>> I also added a test for case-insensitivity. To avoid lots of duplicate
>> boilerplate code, I slightly modified and subclassed the existing test
>> for KeywordAnalyzer.
>>
>> I'd love to see this in the next version of jena-text and Fuseki. Of
>> course I'll rework the patch if necessary. I can also tweak the web
>> documentation to mention this analyzer.
>>
>> -Osma
>>
>>
>> [1]
>> https://github.com/apatry/neo4j-lucene4-index/blob/master/src/main/java/org/neo4j/index/impl/lucene/LowerCaseKeywordAnalyzer.java
>>
>>
>>
>> [2]
>> https://github.com/tuplejump/stargate-core/blob/master/src/main/java/com/tuplejump/stargate/lucene/CaseInsensitiveKeywordAnalyzer.java
>>
>>
>>
>
>


Re: jena-text LowerCaseKeywordAnalyzer implementation

Posted by Osma Suominen <os...@helsinki.fi>.
Hi!

Any interest in my contribution? Anything I can do to help get this, or 
something like it, into jena-text?

-Osma

01.09.2014, 15:38, Osma Suominen kirjoitti:
> Hi!
>
> I liked the option to specify Analyzer for jena-text, as implemented in
> JENA-654. But I'd like to use an analyzer that is otherwise like
> KeywordAnalyzer but case-insensitive, for use in an
> autocomplete/typeahead UI widget. Lucene doesn't include such an
> analyzer, but there are several implementations of the same idea, e.g.
> in neo4j [1] and stargate [2].
>
> I created my own implementation of such an analyzer and added code to
> use it from the assembler. Patch attached.
>
> This analyzer is now in a new package
> org.apache.jena.query.text.analyzer, in case other analyzers for
> jena-text will appear in the future. If you don't like the new package,
> the class can of course be moved to org.apache.jena.query.text.
>
> I also added a test for case-insensitivity. To avoid lots of duplicate
> boilerplate code, I slightly modified and subclassed the existing test
> for KeywordAnalyzer.
>
> I'd love to see this in the next version of jena-text and Fuseki. Of
> course I'll rework the patch if necessary. I can also tweak the web
> documentation to mention this analyzer.
>
> -Osma
>
>
> [1]
> https://github.com/apatry/neo4j-lucene4-index/blob/master/src/main/java/org/neo4j/index/impl/lucene/LowerCaseKeywordAnalyzer.java
>
>
> [2]
> https://github.com/tuplejump/stargate-core/blob/master/src/main/java/com/tuplejump/stargate/lucene/CaseInsensitiveKeywordAnalyzer.java
>
>


-- 
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Teollisuuskatu 23)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi