You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2010/03/06 08:31:27 UTC
[jira] Created: (HADOOP-6623) Add StringUtils.split for non-escaped
single-character separator
Add StringUtils.split for non-escaped single-character separator
----------------------------------------------------------------
Key: HADOOP-6623
URL: https://issues.apache.org/jira/browse/HADOOP-6623
Project: Hadoop Common
Issue Type: Improvement
Components: util
Affects Versions: 0.22.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Minor
This is for HDFS-1028 but useful generally. String.split("/") for example is way slower than an implementation that is specific to only single-character separators.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6623) Add StringUtils.split for
non-escaped single-character separator
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842221#action_12842221 ]
Hadoop QA commented on HADOOP-6623:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12438086/hadoop-6623.txt
against trunk revision 918880.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/412/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/412/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/412/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/412/console
This message is automatically generated.
> Add StringUtils.split for non-escaped single-character separator
> ----------------------------------------------------------------
>
> Key: HADOOP-6623
> URL: https://issues.apache.org/jira/browse/HADOOP-6623
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hadoop-6623.txt
>
>
> This is for HDFS-1028 but useful generally. String.split("/") for example is way slower than an implementation that is specific to only single-character separators.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6623) Add StringUtils.split for non-escaped
single-character separator
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HADOOP-6623:
------------------------------
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Fix Version/s: 0.22.0
Resolution: Fixed
I've just committed this. Thanks Todd!
> Add StringUtils.split for non-escaped single-character separator
> ----------------------------------------------------------------
>
> Key: HADOOP-6623
> URL: https://issues.apache.org/jira/browse/HADOOP-6623
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: hadoop-6623.txt, hadoop-6623.txt
>
>
> This is for HDFS-1028 but useful generally. String.split("/") for example is way slower than an implementation that is specific to only single-character separators.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6623) Add StringUtils.split for non-escaped
single-character separator
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HADOOP-6623:
--------------------------------
Attachment: hadoop-6623.txt
Attached is an updated patch which adds a main() function to TestStringUtils which acts as a benchmark. Benchmark results;
Java impl #4:1339ms
Java impl #5:1095ms
Java impl #6:1257ms
Java impl #7:1386ms
Java impl #8:1470ms
Java impl #9:1467ms
StringUtils impl #4:274ms
StringUtils impl #5:274ms
StringUtils impl #6:274ms
StringUtils impl #7:277ms
StringUtils impl #8:289ms
StringUtils impl #9:291ms
If I double the number of separators in the test string to 10, results are:
Java impl #4:1407ms
Java impl #5:1411ms
Java impl #6:1449ms
Java impl #7:1443ms
Java impl #8:1641ms
Java impl #9:1409ms
StringUtils impl #4:347ms
StringUtils impl #5:347ms
StringUtils impl #6:346ms
StringUtils impl #7:347ms
StringUtils impl #8:355ms
StringUtils impl #9:346ms
> Add StringUtils.split for non-escaped single-character separator
> ----------------------------------------------------------------
>
> Key: HADOOP-6623
> URL: https://issues.apache.org/jira/browse/HADOOP-6623
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hadoop-6623.txt, hadoop-6623.txt
>
>
> This is for HDFS-1028 but useful generally. String.split("/") for example is way slower than an implementation that is specific to only single-character separators.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6623) Add StringUtils.split for non-escaped
single-character separator
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HADOOP-6623:
--------------------------------
Status: Patch Available (was: Open)
> Add StringUtils.split for non-escaped single-character separator
> ----------------------------------------------------------------
>
> Key: HADOOP-6623
> URL: https://issues.apache.org/jira/browse/HADOOP-6623
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hadoop-6623.txt, hadoop-6623.txt
>
>
> This is for HDFS-1028 but useful generally. String.split("/") for example is way slower than an implementation that is specific to only single-character separators.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6623) Add StringUtils.split for non-escaped
single-character separator
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HADOOP-6623:
--------------------------------
Status: Patch Available (was: Open)
> Add StringUtils.split for non-escaped single-character separator
> ----------------------------------------------------------------
>
> Key: HADOOP-6623
> URL: https://issues.apache.org/jira/browse/HADOOP-6623
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hadoop-6623.txt
>
>
> This is for HDFS-1028 but useful generally. String.split("/") for example is way slower than an implementation that is specific to only single-character separators.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6623) Add StringUtils.split for non-escaped
single-character separator
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HADOOP-6623:
--------------------------------
Attachment: hadoop-6623.txt
Simple implementation similar to the existing escapable StringUtils.split. I have not yet benchmarked it, but my guess is that it's faster than String.split.
> Add StringUtils.split for non-escaped single-character separator
> ----------------------------------------------------------------
>
> Key: HADOOP-6623
> URL: https://issues.apache.org/jira/browse/HADOOP-6623
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hadoop-6623.txt
>
>
> This is for HDFS-1028 but useful generally. String.split("/") for example is way slower than an implementation that is specific to only single-character separators.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6623) Add StringUtils.split for non-escaped
single-character separator
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HADOOP-6623:
------------------------------
Status: Open (was: Patch Available)
Looks good. Since this is an optimization, we should really have a benchmark showing a speed improvement.
> Add StringUtils.split for non-escaped single-character separator
> ----------------------------------------------------------------
>
> Key: HADOOP-6623
> URL: https://issues.apache.org/jira/browse/HADOOP-6623
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hadoop-6623.txt
>
>
> This is for HDFS-1028 but useful generally. String.split("/") for example is way slower than an implementation that is specific to only single-character separators.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6623) Add StringUtils.split for
non-escaped single-character separator
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861488#action_12861488 ]
Hadoop QA commented on HADOOP-6623:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12442970/hadoop-6623.txt
against trunk revision 938590.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/483/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/483/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/483/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/483/console
This message is automatically generated.
> Add StringUtils.split for non-escaped single-character separator
> ----------------------------------------------------------------
>
> Key: HADOOP-6623
> URL: https://issues.apache.org/jira/browse/HADOOP-6623
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hadoop-6623.txt, hadoop-6623.txt
>
>
> This is for HDFS-1028 but useful generally. String.split("/") for example is way slower than an implementation that is specific to only single-character separators.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.