You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org> on 2008/05/28 23:06:45 UTC
[jira] Created: (HADOOP-3461) Remove dfs.StringBytesWritbale
Remove dfs.StringBytesWritbale
------------------------------
Key: HADOOP-3461
URL: https://issues.apache.org/jira/browse/HADOOP-3461
Project: Hadoop Core
Issue Type: Improvement
Components: dfs
Reporter: Tsz Wo (Nicholas), SZE
In dfs, some classes, such as INodeFileUnderConstruction use, StringBytesWritbale as member type. The values are converted from String to StringBytesWritbale in setter and converted back to String in getter. These overhead is unnecessary.
StringBytesWritbale is not used in RPC nor dfs image. No version change will be introduced.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3461) Remove dfs.StringBytesWritbale
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644480#action_12644480 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3461:
------------------------------------------------
There are ~10% improvement for NNThroughputBenchmark -op create.
> Remove dfs.StringBytesWritbale
> ------------------------------
>
> Key: HADOOP-3461
> URL: https://issues.apache.org/jira/browse/HADOOP-3461
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.20.0
>
> Attachments: 3461_20081030.patch
>
>
> In dfs, some classes, such as INodeFileUnderConstruction use, StringBytesWritbale as member type. The values are converted from String to StringBytesWritbale in setter and converted back to String in getter. These overhead is unnecessary.
> StringBytesWritbale is not used in RPC nor dfs image. No version change will be introduced.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3461) Remove dfs.StringBytesWritbale
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644466#action_12644466 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3461:
------------------------------------------------
{noformat}
[exec] -1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
{noformat}
All unit tests are passed locally.
> Remove dfs.StringBytesWritbale
> ------------------------------
>
> Key: HADOOP-3461
> URL: https://issues.apache.org/jira/browse/HADOOP-3461
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: 3461_20081030.patch
>
>
> In dfs, some classes, such as INodeFileUnderConstruction use, StringBytesWritbale as member type. The values are converted from String to StringBytesWritbale in setter and converted back to String in getter. These overhead is unnecessary.
> StringBytesWritbale is not used in RPC nor dfs image. No version change will be introduced.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3461) Remove dfs.StringBytesWritbale
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644465#action_12644465 ]
Konstantin Shvachko commented on HADOOP-3461:
---------------------------------------------
We have a time/space tradeoff here. Using Strings eliminates conversions to/from StringBytesWritable, but it also increases the size of INodeFileUnderConstruction since String uses 2 bytes per character instead of 1.
I think it is right to resolve it in favor of time because the number of files under construction is small relative to the total number of files.
+1
> Remove dfs.StringBytesWritbale
> ------------------------------
>
> Key: HADOOP-3461
> URL: https://issues.apache.org/jira/browse/HADOOP-3461
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: 3461_20081030.patch
>
>
> In dfs, some classes, such as INodeFileUnderConstruction use, StringBytesWritbale as member type. The values are converted from String to StringBytesWritbale in setter and converted back to String in getter. These overhead is unnecessary.
> StringBytesWritbale is not used in RPC nor dfs image. No version change will be introduced.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3461) Remove dfs.StringBytesWritbale
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645245#action_12645245 ]
Hudson commented on HADOOP-3461:
--------------------------------
Integrated in Hadoop-trunk #652 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/652/])
> Remove dfs.StringBytesWritbale
> ------------------------------
>
> Key: HADOOP-3461
> URL: https://issues.apache.org/jira/browse/HADOOP-3461
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.20.0
>
> Attachments: 3461_20081030.patch
>
>
> In dfs, some classes, such as INodeFileUnderConstruction use, StringBytesWritbale as member type. The values are converted from String to StringBytesWritbale in setter and converted back to String in getter. These overhead is unnecessary.
> StringBytesWritbale is not used in RPC nor dfs image. No version change will be introduced.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3461) Remove dfs.StringBytesWritbale
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3461:
-------------------------------------------
Assignee: Tsz Wo (Nicholas), SZE
Status: Patch Available (was: Open)
> Remove dfs.StringBytesWritbale
> ------------------------------
>
> Key: HADOOP-3461
> URL: https://issues.apache.org/jira/browse/HADOOP-3461
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: 3461_20081030.patch
>
>
> In dfs, some classes, such as INodeFileUnderConstruction use, StringBytesWritbale as member type. The values are converted from String to StringBytesWritbale in setter and converted back to String in getter. These overhead is unnecessary.
> StringBytesWritbale is not used in RPC nor dfs image. No version change will be introduced.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3461) Remove dfs.StringBytesWritbale
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3461:
-------------------------------------------
Attachment: 3461_20081030.patch
3461_20081030.patch: remove StringBytesWritable.
Note that StringBytesWritable is neither used in images/edits nor rpc. So there is no layout or protocol changes.
> Remove dfs.StringBytesWritbale
> ------------------------------
>
> Key: HADOOP-3461
> URL: https://issues.apache.org/jira/browse/HADOOP-3461
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Attachments: 3461_20081030.patch
>
>
> In dfs, some classes, such as INodeFileUnderConstruction use, StringBytesWritbale as member type. The values are converted from String to StringBytesWritbale in setter and converted back to String in getter. These overhead is unnecessary.
> StringBytesWritbale is not used in RPC nor dfs image. No version change will be introduced.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3461) Remove dfs.StringBytesWritbale
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3461:
-------------------------------------------
Resolution: Fixed
Fix Version/s: 0.20.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this.
> Remove dfs.StringBytesWritbale
> ------------------------------
>
> Key: HADOOP-3461
> URL: https://issues.apache.org/jira/browse/HADOOP-3461
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.20.0
>
> Attachments: 3461_20081030.patch
>
>
> In dfs, some classes, such as INodeFileUnderConstruction use, StringBytesWritbale as member type. The values are converted from String to StringBytesWritbale in setter and converted back to String in getter. These overhead is unnecessary.
> StringBytesWritbale is not used in RPC nor dfs image. No version change will be introduced.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.