You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "lfoppiano (via GitHub)" <gi...@apache.org> on 2026/06/08 09:21:01 UTC

[PR] NUTCH-3180 BasicURLNormalizer missing catching ICUInputTooLongException [nutch]

lfoppiano opened a new pull request, #923:
URL: https://github.com/apache/nutch/pull/923

   This PR fixes the issue https://issues.apache.org/jira/browse/NUTCH-3180. 
   It hardens Nutch’s IDNA2008 hostname conversion so that unchecked ICU exceptions (e.g., from UTS46/Punycode) are converted into MalformedURLException, allowing callers like BasicURLNormalizer/URLCleaner to reject bad URLs instead of crashing a task.
   
   Changes:
     -  Wrap ICU IDNA.nameToASCII/nameToUnicode in URLUtil.convertIDNA2008 with a try/catch to translate unchecked ICU/UTS46/Punycode exceptions into MalformedURLException (with cause attached).
     -  Add regression tests intended to cover possible Exception thrown by the underlying library 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@nutch.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NUTCH-3180 BasicURLNormalizer missing catching ICUInputTooLongException [nutch]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #923:
URL: https://github.com/apache/nutch/pull/923#issuecomment-4677825814

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=apache_nutch&pullRequest=923) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [B Maintainability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_nutch&pullRequest=923) (required ≥ A)  
     
   <!-- sqra:qg_revision=2f45992bef0c6a3cbc7fcb61362014f4e87c9bf35abeafb84b56be16889439c0 -->
   <!-- sqra-placement-anchor -->
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_nutch&pullRequest=923)
   
   ##   
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png '') Catch issues before they fail your Quality Gate with our IDE extension ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png '') [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@nutch.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NUTCH-3180 BasicURLNormalizer missing catching ICUInputTooLongException [nutch]

Posted by "sebastian-nagel (via GitHub)" <gi...@apache.org>.
sebastian-nagel merged PR #923:
URL: https://github.com/apache/nutch/pull/923


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@nutch.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NUTCH-3180 BasicURLNormalizer missing catching ICUInputTooLongException [nutch]

Posted by "lfoppiano (via GitHub)" <gi...@apache.org>.
lfoppiano commented on PR #923:
URL: https://github.com/apache/nutch/pull/923#issuecomment-4669047416

   @sebastian-nagel should I fix the issue with sonaqubecloud? I'm not sure what would be the preferred way to fix it. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@nutch.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NUTCH-3180 BasicURLNormalizer missing catching ICUInputTooLongException [nutch]

Posted by "lfoppiano (via GitHub)" <gi...@apache.org>.
lfoppiano commented on PR #923:
URL: https://github.com/apache/nutch/pull/923#issuecomment-4671811751

   OK. Removed the log. I though it would be useful for this specific issue, but indeed it's an antipattern 😄 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@nutch.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NUTCH-3180 BasicURLNormalizer missing catching ICUInputTooLongException [nutch]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #923:
URL: https://github.com/apache/nutch/pull/923#issuecomment-4690312375

   ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=apache_nutch&pullRequest=923) **Quality Gate passed**  
   Issues  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 New issues](https://sonarcloud.io/project/issues?id=apache_nutch&pullRequest=923&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_nutch&pullRequest=923&issueStatuses=ACCEPTED)
   
   Measures  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_nutch&pullRequest=923&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [100.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_nutch&pullRequest=923&metric=new_coverage&view=list)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_nutch&pullRequest=923&metric=new_duplicated_lines_density&view=list)  
     
   <!-- sqra-placement-anchor -->
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_nutch&pullRequest=923)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@nutch.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NUTCH-3180 BasicURLNormalizer missing catching ICUInputTooLongException [nutch]

Posted by "lfoppiano (via GitHub)" <gi...@apache.org>.
lfoppiano commented on PR #923:
URL: https://github.com/apache/nutch/pull/923#issuecomment-4687766943

   @sebastian-nagel sorry I forgot it. Now the second log is also gone. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@nutch.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NUTCH-3180 BasicURLNormalizer missing catching ICUInputTooLongException [nutch]

Posted by "sebastian-nagel (via GitHub)" <gi...@apache.org>.
sebastian-nagel commented on PR #923:
URL: https://github.com/apache/nutch/pull/923#issuecomment-4671731795

   > should I fix the issue with sonaqubecloud?
   >> "Either log this exception and handle it, or rethrow it with some contextual information."
   
   You might remove the debug log - it's on you. The original exception is preserved per `initCause(e)`, and most callers log the MalformedURLException anyway. But this is really a minor issue - there are many occurrences in the Nutch codebase where an error is logged and then thrown as exception. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@nutch.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NUTCH-3180 BasicURLNormalizer missing catching ICUInputTooLongException [nutch]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #923:
URL: https://github.com/apache/nutch/pull/923#issuecomment-4667467887

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=apache_nutch&pullRequest=923) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [B Maintainability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_nutch&pullRequest=923) (required ≥ A)  
     
   <!-- sqra:qg_revision=2f45992bef0c6a3cbc7fcb61362014f4e87c9bf35abeafb84b56be16889439c0 -->
   <!-- sqra-placement-anchor -->
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_nutch&pullRequest=923)
   
   ##   
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png '') Catch issues before they fail your Quality Gate with our IDE extension ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png '') [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@nutch.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NUTCH-3180 BasicURLNormalizer missing catching ICUInputTooLongException [nutch]

Posted by "sebastian-nagel (via GitHub)" <gi...@apache.org>.
sebastian-nagel commented on PR #923:
URL: https://github.com/apache/nutch/pull/923#issuecomment-4682667345

   > Removed the log.
   
   @lfoppiano: there is a second logging statement, it's still in the code. Do you want to remove it?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@nutch.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org