You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Koji Noguchi (JIRA)" <ji...@apache.org> on 2007/06/19 22:41:25 UTC
[jira] Created: (HADOOP-1506) distcp not preserving the replication
factor and block size of source files
distcp not preserving the replication factor and block size of source files
---------------------------------------------------------------------------
Key: HADOOP-1506
URL: https://issues.apache.org/jira/browse/HADOOP-1506
Project: Hadoop
Issue Type: Bug
Components: util
Affects Versions: 0.12.3
Reporter: Koji Noguchi
Priority: Minor
Myabe not a bug but a feature request.
It would be nice if the source file and the target file have the same replication factor and block size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-1506) distcp not preserving the
replication factor and block size of source files
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley resolved HADOOP-1506.
-----------------------------------
Resolution: Duplicate
Fix Version/s: 0.15.0
Fixed by HADOOP-1569.
> distcp not preserving the replication factor and block size of source files
> ---------------------------------------------------------------------------
>
> Key: HADOOP-1506
> URL: https://issues.apache.org/jira/browse/HADOOP-1506
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.12.3
> Reporter: Koji Noguchi
> Assignee: Chris Douglas
> Priority: Minor
> Fix For: 0.15.0
>
>
> Myabe not a bug but a feature request.
> It would be nice if the source file and the target file have the same replication factor and block size.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1506) distcp not preserving the
replication factor and block size of source files
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas reassigned HADOOP-1506:
-------------------------------------
Assignee: Chris Douglas
> distcp not preserving the replication factor and block size of source files
> ---------------------------------------------------------------------------
>
> Key: HADOOP-1506
> URL: https://issues.apache.org/jira/browse/HADOOP-1506
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.12.3
> Reporter: Koji Noguchi
> Assignee: Chris Douglas
> Priority: Minor
>
> Myabe not a bug but a feature request.
> It would be nice if the source file and the target file have the same replication factor and block size.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1506) distcp not preserving the
replication factor and block size of source files
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12511324 ]
Raghu Angadi commented on HADOOP-1506:
--------------------------------------
This should probably be optional. DFS in general wants to enforce the condition that dfs.block.size should be a multiple of io.bytes.per.checksum. This could be an issue if we are copying files written in earlier versions. Making this optional also has the advantage that we could change block size etc if we want to.
> distcp not preserving the replication factor and block size of source files
> ---------------------------------------------------------------------------
>
> Key: HADOOP-1506
> URL: https://issues.apache.org/jira/browse/HADOOP-1506
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.12.3
> Reporter: Koji Noguchi
> Priority: Minor
>
> Myabe not a bug but a feature request.
> It would be nice if the source file and the target file have the same replication factor and block size.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1506) distcp not preserving the
replication factor and block size of source files
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517669 ]
Konstantin Shvachko commented on HADOOP-1506:
---------------------------------------------
What is the reason for keeping the same block size in the target file?
The target file system may have a different default block size.
Why do we want to go against the default in this case?
Ex. If we copy a file from ext2 with 1K blocks to ext3 with 8K blocks we are not trying to preserve any block sizes.
> distcp not preserving the replication factor and block size of source files
> ---------------------------------------------------------------------------
>
> Key: HADOOP-1506
> URL: https://issues.apache.org/jira/browse/HADOOP-1506
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.12.3
> Reporter: Koji Noguchi
> Priority: Minor
>
> Myabe not a bug but a feature request.
> It would be nice if the source file and the target file have the same replication factor and block size.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1506) distcp not preserving the
replication factor and block size of source files
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12506621 ]
dhruba borthakur commented on HADOOP-1506:
------------------------------------------
We could avoid using FileSystem.getReplication and FileSystem.getBlockSize(). They will result in separate RPCs to namenode. Instead, we can get getFileSystus introduced by HADOOP-1377. Or use FileSystem.listPaths because it caches file attributes in the Path object.
> distcp not preserving the replication factor and block size of source files
> ---------------------------------------------------------------------------
>
> Key: HADOOP-1506
> URL: https://issues.apache.org/jira/browse/HADOOP-1506
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.12.3
> Reporter: Koji Noguchi
> Priority: Minor
>
> Myabe not a bug but a feature request.
> It would be nice if the source file and the target file have the same replication factor and block size.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.