You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Todd Feak (JIRA)" <ji...@apache.org> on 2008/10/17 20:05:44 UTC
[jira] Created: (SOLR-814) Add new Japanese Hiragana Filter and
Factory
Add new Japanese Hiragana Filter and Factory
--------------------------------------------
Key: SOLR-814
URL: https://issues.apache.org/jira/browse/SOLR-814
Project: Solr
Issue Type: New Feature
Components: search
Reporter: Todd Feak
Priority: Minor
Japanese Hiragana and Katakana character sets can be easily translated between. This filter normalizes all Hiragana characters to their Katakana counterpart, allowing for indexing and searching using either.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-814) Add new Japanese Hiragana Filter and
Factory
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Sekiguchi updated SOLR-814:
--------------------------------
Component/s: (was: search)
Analysis
> Add new Japanese Hiragana Filter and Factory
> --------------------------------------------
>
> Key: SOLR-814
> URL: https://issues.apache.org/jira/browse/SOLR-814
> Project: Solr
> Issue Type: New Feature
> Components: Analysis
> Affects Versions: 1.3
> Reporter: Todd Feak
> Assignee: Koji Sekiguchi
> Priority: Minor
> Attachments: SOLR-814.patch
>
>
> Japanese Hiragana and Katakana character sets can be easily translated between. This filter normalizes all Hiragana characters to their Katakana counterpart, allowing for indexing and searching using either.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-814) Add new Japanese Hiragana Filter and
Factory
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Sekiguchi resolved SOLR-814.
---------------------------------
Resolution: Won't Fix
Using CharFilter can solve this problem in a flexible manner.
> Add new Japanese Hiragana Filter and Factory
> --------------------------------------------
>
> Key: SOLR-814
> URL: https://issues.apache.org/jira/browse/SOLR-814
> Project: Solr
> Issue Type: New Feature
> Components: Analysis
> Affects Versions: 1.3
> Reporter: Todd Feak
> Assignee: Koji Sekiguchi
> Priority: Minor
> Attachments: SOLR-814.patch
>
>
> Japanese Hiragana and Katakana character sets can be easily translated between. This filter normalizes all Hiragana characters to their Katakana counterpart, allowing for indexing and searching using either.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-814) Add new Japanese Hiragana Filter and
Factory
Posted by "Todd Feak (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640615#action_12640615 ]
Todd Feak commented on SOLR-814:
--------------------------------
Yes, they are used differently.
However, a word written in Hiragana is the *same* word when written in Katakana. Same meaning. Futhermore, it's not always cut and dried which to use. For example, a movie title may be written in Hiragana or Katakana, depending on the Director's preference. The user (searcher) may not have remembered the Director's preference, so may search using the other. Without this normalization they would get a search miss.
I don't doubt your experience at Ultraseek, but this feature was explicitly asked for by Japanese (native speaking) engineers at Sony. I *just* (literally) double checked with a couple of onsite native speaking Japanese engineers and both agree that this is useful, at least for our searches.
I would say that it should be up to the schema developer as to whether this functionality is useful or not for their situation. Either way, I offer it up to the community for their decision.
> Add new Japanese Hiragana Filter and Factory
> --------------------------------------------
>
> Key: SOLR-814
> URL: https://issues.apache.org/jira/browse/SOLR-814
> Project: Solr
> Issue Type: New Feature
> Components: search
> Affects Versions: 1.3
> Reporter: Todd Feak
> Priority: Minor
> Attachments: SOLR-814.patch
>
>
> Japanese Hiragana and Katakana character sets can be easily translated between. This filter normalizes all Hiragana characters to their Katakana counterpart, allowing for indexing and searching using either.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-814) Add new Japanese Hiragana Filter and
Factory
Posted by "Todd Feak (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Feak updated SOLR-814:
---------------------------
Affects Version/s: 1.3
> Add new Japanese Hiragana Filter and Factory
> --------------------------------------------
>
> Key: SOLR-814
> URL: https://issues.apache.org/jira/browse/SOLR-814
> Project: Solr
> Issue Type: New Feature
> Components: search
> Affects Versions: 1.3
> Reporter: Todd Feak
> Priority: Minor
>
> Japanese Hiragana and Katakana character sets can be easily translated between. This filter normalizes all Hiragana characters to their Katakana counterpart, allowing for indexing and searching using either.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-814) Add new Japanese Hiragana Filter and
Factory
Posted by "Walter Underwood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640605#action_12640605 ]
Walter Underwood commented on SOLR-814:
---------------------------------------
This seems like a bad idea. Hirigana and katakana are used quite differently in Japanese. They are not interchangeable.
I was the engineer for Japanese support in Ultraseek for years and even visited our distributor there, but no one ever asked for this feature. They asked for a lot of things, but never this.
It is very useful, maybe essential, to normalize full-width and half-width versions of hirigana, katakana, and ASCII.
> Add new Japanese Hiragana Filter and Factory
> --------------------------------------------
>
> Key: SOLR-814
> URL: https://issues.apache.org/jira/browse/SOLR-814
> Project: Solr
> Issue Type: New Feature
> Components: search
> Affects Versions: 1.3
> Reporter: Todd Feak
> Priority: Minor
> Attachments: SOLR-814.patch
>
>
> Japanese Hiragana and Katakana character sets can be easily translated between. This filter normalizes all Hiragana characters to their Katakana counterpart, allowing for indexing and searching using either.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-814) Add new Japanese Hiragana Filter and
Factory
Posted by "Todd Feak (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Feak updated SOLR-814:
---------------------------
Attachment: SOLR-814.patch
Attached patch containing Filter, Factory, and Unit Tests for both
> Add new Japanese Hiragana Filter and Factory
> --------------------------------------------
>
> Key: SOLR-814
> URL: https://issues.apache.org/jira/browse/SOLR-814
> Project: Solr
> Issue Type: New Feature
> Components: search
> Affects Versions: 1.3
> Reporter: Todd Feak
> Priority: Minor
> Attachments: SOLR-814.patch
>
>
> Japanese Hiragana and Katakana character sets can be easily translated between. This filter normalizes all Hiragana characters to their Katakana counterpart, allowing for indexing and searching using either.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-814) Add new Japanese Hiragana Filter and
Factory
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Sekiguchi reassigned SOLR-814:
-----------------------------------
Assignee: Koji Sekiguchi
> Add new Japanese Hiragana Filter and Factory
> --------------------------------------------
>
> Key: SOLR-814
> URL: https://issues.apache.org/jira/browse/SOLR-814
> Project: Solr
> Issue Type: New Feature
> Components: search
> Affects Versions: 1.3
> Reporter: Todd Feak
> Assignee: Koji Sekiguchi
> Priority: Minor
> Attachments: SOLR-814.patch
>
>
> Japanese Hiragana and Katakana character sets can be easily translated between. This filter normalizes all Hiragana characters to their Katakana counterpart, allowing for indexing and searching using either.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.