You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org> on 2008/05/06 01:35:55 UTC
[jira] Created: (HADOOP-3347) FSEditLog should not writes long and
short as UTF8 and should not use ArrayWritable for writing non-array items
FSEditLog should not writes long and short as UTF8 and should not use ArrayWritable for writing non-array items
---------------------------------------------------------------------------------------------------------------
Key: HADOOP-3347
URL: https://issues.apache.org/jira/browse/HADOOP-3347
Project: Hadoop Core
Issue Type: Improvement
Components: dfs
Reporter: Tsz Wo (Nicholas), SZE
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3347) FSEditLog should not writes long and
short as UTF8 and should not use ArrayWritable for writing non-array items
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3347:
-------------------------------------------
Description:
In FSEditLog,
- long and short are first converted to String and are further converted to UTF8
- For some non-array items, it first create an ArrayWritable object to hold all the items and then writes the ArrayWritable object.
These result creating many intermediate objects which affects Namenode CPU performance and Namenode restart.
> FSEditLog should not writes long and short as UTF8 and should not use ArrayWritable for writing non-array items
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3347
> URL: https://issues.apache.org/jira/browse/HADOOP-3347
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
>
> In FSEditLog,
> - long and short are first converted to String and are further converted to UTF8
> - For some non-array items, it first create an ArrayWritable object to hold all the items and then writes the ArrayWritable object.
> These result creating many intermediate objects which affects Namenode CPU performance and Namenode restart.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3347) FSEditLog should not writes long
and short as UTF8 and should not use ArrayWritable for writing non-array
items
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12594400#action_12594400 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3347:
------------------------------------------------
{code}
static UTF8 toLogLong(long timestamp) {
return new UTF8(Long.toString(timestamp));
}
void logRename(String src, String dst, long timestamp) {
UTF8 info[] = new UTF8[] {
new UTF8(src),
new UTF8(dst),
FSEditLog.toLogLong(timestamp)};
logEdit(OP_RENAME, new ArrayWritable(UTF8.class, info));
}
{code}
For example, logRename converts timestamp from long to UTF8. Then, it creates an ArrayWritable to hold everything. A better way would be using LongWritable and writing everything directly without creating an ArrayWritable.
> FSEditLog should not writes long and short as UTF8 and should not use ArrayWritable for writing non-array items
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3347
> URL: https://issues.apache.org/jira/browse/HADOOP-3347
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.