[jira] [Commented] (NUTCH-1321) IDNNormalizer

["İlhami KALKAN (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NUTCH-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843047#comment-13843047 ] 

İlhami KALKAN commented on NUTCH-1321:
--------------------------------------

Hi Markus.
I have a question for IDN. I have urls like http://www.çevir.com. In inject and parse phases, they are rejected by URLFilters. Besides, I want to index them as unicode like www.çevir.com. How can I do this? By fixing IDNNormalizer plugin patch? I think, convert them to punycode in inject and parse phase then force revert to unicode url to index. Am I wrong or do you have any recommendation.
I addition, toUNICODE method does not work correctly. When I try to convert http://www.çevir.com with this method, it return java.net.URISyntaxException. If we dont use URI object, there is no problem. Why do we need URI object in toUNICODE method? 

> IDNNormalizer
> -------------
>
>                 Key: NUTCH-1321
>                 URL: https://issues.apache.org/jira/browse/NUTCH-1321
>             Project: Nutch
>          Issue Type: New Feature
>            Reporter: Markus Jelsma
>            Assignee: Markus Jelsma
>             Fix For: 1.9
>
>         Attachments: Nutch-1321.patch
>
>
> Right now, IDN's are indexed as ASCII. An IDNNormalizer is to be used with an indexer so it will encode ASCII URL's to their proper unicode equivalant.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)