You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Christian Kunz (JIRA)" <ji...@apache.org> on 2007/11/13 02:51:50 UTC
[jira] Updated: (HADOOP-713) dfs list operation is too expensive
[ https://issues.apache.org/jira/browse/HADOOP-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christian Kunz updated HADOOP-713:
----------------------------------
Priority: Blocker (was: Major)
Changing to blocker based on conversation with Sameer.
> dfs list operation is too expensive
> -----------------------------------
>
> Key: HADOOP-713
> URL: https://issues.apache.org/jira/browse/HADOOP-713
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.8.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Priority: Blocker
>
> A list request to dfs returns an array of DFSFileInfo. A DFSFileInfo of a directory contains a field called contentsLen, indicating its size which gets computed at the namenode side by resursively going through its subdirs. At the same time, the whole dfs directory tree is locked.
> The list operation is used a lot by DFSClient for listing a directory, getting a file's size and # of replicas, and getting the size of dfs. Only the last operation needs the field contentsLen to be computed.
> To reduce its cost, we can add a flag to the list request. ContentsLen is computed If the flag is set. By default, the flag is false.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
RE: [jira] Updated: (HADOOP-713) dfs list operation is too expensive
Posted by Christian Kunz <ck...@yahoo-inc.com>.
Assigned it to 0.15.1
-Christian
-----Original Message-----
From: Nigel Daley [mailto:ndaley@yahoo-inc.com]
Sent: Monday, November 12, 2007 11:57 PM
To: hadoop-dev@lucene.apache.org
Subject: Re: [jira] Updated: (HADOOP-713) dfs list operation is too
expensive
Blocker for what release? Please assign to a release.
On Nov 12, 2007, at 5:51 PM, Christian Kunz (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/HADOOP-713?
> page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Christian Kunz updated HADOOP-713:
> ----------------------------------
>
> Priority: Blocker (was: Major)
>
> Changing to blocker based on conversation with Sameer.
>
>> dfs list operation is too expensive
>> -----------------------------------
>>
>> Key: HADOOP-713
>> URL: https://issues.apache.org/jira/browse/HADOOP-713
>> Project: Hadoop
>> Issue Type: Improvement
>> Components: dfs
>> Affects Versions: 0.8.0
>> Reporter: Hairong Kuang
>> Assignee: Hairong Kuang
>> Priority: Blocker
>>
>> A list request to dfs returns an array of DFSFileInfo. A DFSFileInfo
>> of a directory contains a field called contentsLen, indicating its
>> size which gets computed at the namenode side by resursively going
>> through its subdirs. At the same time, the whole dfs directory tree
>> is locked.
>> The list operation is used a lot by DFSClient for listing a
>> directory, getting a file's size and # of replicas, and getting the
>> size of dfs. Only the last operation needs the field contentsLen to
>> be computed.
>> To reduce its cost, we can add a flag to the list request.
>> ContentsLen is computed If the flag is set. By default, the flag is
>> false.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
Re: [jira] Updated: (HADOOP-713) dfs list operation is too expensive
Posted by Nigel Daley <nd...@yahoo-inc.com>.
Blocker for what release? Please assign to a release.
On Nov 12, 2007, at 5:51 PM, Christian Kunz (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/HADOOP-713?
> page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Christian Kunz updated HADOOP-713:
> ----------------------------------
>
> Priority: Blocker (was: Major)
>
> Changing to blocker based on conversation with Sameer.
>
>> dfs list operation is too expensive
>> -----------------------------------
>>
>> Key: HADOOP-713
>> URL: https://issues.apache.org/jira/browse/HADOOP-713
>> Project: Hadoop
>> Issue Type: Improvement
>> Components: dfs
>> Affects Versions: 0.8.0
>> Reporter: Hairong Kuang
>> Assignee: Hairong Kuang
>> Priority: Blocker
>>
>> A list request to dfs returns an array of DFSFileInfo. A
>> DFSFileInfo of a directory contains a field called contentsLen,
>> indicating its size which gets computed at the namenode side by
>> resursively going through its subdirs. At the same time, the whole
>> dfs directory tree is locked.
>> The list operation is used a lot by DFSClient for listing a
>> directory, getting a file's size and # of replicas, and getting
>> the size of dfs. Only the last operation needs the field
>> contentsLen to be computed.
>> To reduce its cost, we can add a flag to the list request.
>> ContentsLen is computed If the flag is set. By default, the flag
>> is false.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>