You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "dhruba borthakur (JIRA)" <ji...@apache.org> on 2007/12/02 11:05:43 UTC
[jira] Created: (HADOOP-2330) Preallocate transaction log to
improve namenode transaction logging performance
Preallocate transaction log to improve namenode transaction logging performance
-------------------------------------------------------------------------------
Key: HADOOP-2330
URL: https://issues.apache.org/jira/browse/HADOOP-2330
Project: Hadoop
Issue Type: Improvement
Components: dfs
Reporter: dhruba borthakur
Assignee: dhruba borthakur
In the current implementation, the transaction is opened in "append" mode and evenry new transaction is written to the end of the log. This means that new blocks get allocated to the edits file frequently.
It is worth measuring the performance improvement when big chunks of the transaction log are allocated up front. Adding new transactions do not cause frequent block allocations for the edits log.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2330) Preallocate transaction log to
improve namenode transaction logging performance
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-2330:
-------------------------------------
Attachment: preallocateTransactionLog.patch
Preallocate edits log in chunks of 1MB.
> Preallocate transaction log to improve namenode transaction logging performance
> -------------------------------------------------------------------------------
>
> Key: HADOOP-2330
> URL: https://issues.apache.org/jira/browse/HADOOP-2330
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Attachments: preallocateTransactionLog.patch
>
>
> In the current implementation, the transaction log is opened in "append" mode and every new transaction is written to the end of the log. This means that new blocks get allocated to the edits file frequently.
> It is worth measuring the performance improvement when big chunks of the transaction log are allocated up front. Adding new transactions do not cause frequent block allocations for the edits log.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2330) Preallocate transaction log to
improve namenode transaction logging performance
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-2330:
-------------------------------------
Description:
In the current implementation, the transaction log is opened in "append" mode and every new transaction is written to the end of the log. This means that new blocks get allocated to the edits file frequently.
It is worth measuring the performance improvement when big chunks of the transaction log are allocated up front. Adding new transactions do not cause frequent block allocations for the edits log.
was:
In the current implementation, the transaction is opened in "append" mode and evenry new transaction is written to the end of the log. This means that new blocks get allocated to the edits file frequently.
It is worth measuring the performance improvement when big chunks of the transaction log are allocated up front. Adding new transactions do not cause frequent block allocations for the edits log.
> Preallocate transaction log to improve namenode transaction logging performance
> -------------------------------------------------------------------------------
>
> Key: HADOOP-2330
> URL: https://issues.apache.org/jira/browse/HADOOP-2330
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
>
> In the current implementation, the transaction log is opened in "append" mode and every new transaction is written to the end of the log. This means that new blocks get allocated to the edits file frequently.
> It is worth measuring the performance improvement when big chunks of the transaction log are allocated up front. Adding new transactions do not cause frequent block allocations for the edits log.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.