You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Philip Zeyliger (JIRA)" <ji...@apache.org> on 2012/09/04 16:05:07 UTC
[jira] [Created] (HADOOP-8761) Help for FsShell's Stat incorrectly
mentions "file size in blocks" (should be bytes)
Philip Zeyliger created HADOOP-8761:
---------------------------------------
Summary: Help for FsShell's Stat incorrectly mentions "file size in blocks" (should be bytes)
Key: HADOOP-8761
URL: https://issues.apache.org/jira/browse/HADOOP-8761
Project: Hadoop Common
Issue Type: Bug
Components: fs
Reporter: Philip Zeyliger
Priority: Minor
Trivial patch attached corrects the usage information. Stat.java calls FileStatus.getLen(), which is most definitely the file size in bytes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8761) Help for FsShell's Stat
incorrectly mentions "file size in blocks" (should be bytes)
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447840#comment-13447840 ]
Andy Isaacson commented on HADOOP-8761:
---------------------------------------
bq. I think the correct behavior should be to match the usage. The unix stat uses %b for blocks, and %s for size in bytes.
I agree that adding %s for size in bytes is a good idea, but changing the semantics of %b is an incompatible change, which seems risky. OTOH, I think it would be potentially very useful to expose "number of blocks in this file".
Tangentially, the documentation also fails to describe %F.
> Help for FsShell's Stat incorrectly mentions "file size in blocks" (should be bytes)
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-8761
> URL: https://issues.apache.org/jira/browse/HADOOP-8761
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Priority: Minor
> Attachments: HADOOP-8761.patch.txt
>
>
> Trivial patch attached corrects the usage information. Stat.java calls FileStatus.getLen(), which is most definitely the file size in bytes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8761) Help for FsShell's Stat
incorrectly mentions "file size in blocks" (should be bytes)
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447989#comment-13447989 ]
Daryn Sharp commented on HADOOP-8761:
-------------------------------------
True, it's incompatible yet correct behavior. I guess our options are codify the bad behavior forever, or fix it now. I lean toward the latter since there's enough churn in 2.x that this will be a minor inconvenience for people to adjust. What do the other watchers/lurkers think?
> Help for FsShell's Stat incorrectly mentions "file size in blocks" (should be bytes)
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-8761
> URL: https://issues.apache.org/jira/browse/HADOOP-8761
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Priority: Minor
> Attachments: HADOOP-8761.patch.txt
>
>
> Trivial patch attached corrects the usage information. Stat.java calls FileStatus.getLen(), which is most definitely the file size in bytes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8761) Help for FsShell's Stat
incorrectly mentions "file size in blocks" (should be bytes)
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447810#comment-13447810 ]
Daryn Sharp commented on HADOOP-8761:
-------------------------------------
I think the correct behavior should be to match the usage. The unix stat uses %b for blocks, and %s for size in bytes.
> Help for FsShell's Stat incorrectly mentions "file size in blocks" (should be bytes)
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-8761
> URL: https://issues.apache.org/jira/browse/HADOOP-8761
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Priority: Minor
> Attachments: HADOOP-8761.patch.txt
>
>
> Trivial patch attached corrects the usage information. Stat.java calls FileStatus.getLen(), which is most definitely the file size in bytes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8761) Help for FsShell's Stat incorrectly
mentions "file size in blocks" (should be bytes)
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Philip Zeyliger updated HADOOP-8761:
------------------------------------
Attachment: HADOOP-8761.patch.txt
Trivial patch attached.
> Help for FsShell's Stat incorrectly mentions "file size in blocks" (should be bytes)
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-8761
> URL: https://issues.apache.org/jira/browse/HADOOP-8761
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Reporter: Philip Zeyliger
> Priority: Minor
> Attachments: HADOOP-8761.patch.txt
>
>
> Trivial patch attached corrects the usage information. Stat.java calls FileStatus.getLen(), which is most definitely the file size in bytes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8761) Help for FsShell's Stat
incorrectly mentions "file size in blocks" (should be bytes)
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448102#comment-13448102 ]
Andy Isaacson commented on HADOOP-8761:
---------------------------------------
bq. True, it's incompatible yet correct behavior. I guess our options are codify the bad behavior forever, or fix it now.
Since this is client-side code, we could do something like
# in 2.1.0, add %s and leave %b giving bytes, with a stderr message that %b will change in a future release.
# in 2.2.0, change %b to give blocks and remove the stderr message.
bq. True, it's incompatible yet correct behavior.
I'm a little skeptical that "correct" with respect to POSIX stat(2) is significant here. The "blocks" used in {{struct stat}} are very different from the "blocks" in HDFS; POSIX blocks are fixed size, atomically written [1], and the block size is a legacy feature (hardcoded to 512 bytes which is not the underlying size of anything anymore). By contrast HDFS blocks are variable size, nonatomic, and are much larger than POSIX blocks.
I agree that exposing the blocksize (and the blockcount) is a pretty valuable feature, but there's a lot of caveats. Just off the top of my head: a single file can have multiple blocksizes.
[1] blocks aren't guaranteed to be atomic by the POSIX spec AFAIK, but as a practical matter modern implementations are atomic at some blocksize between 512 and 4096 bytes.
> Help for FsShell's Stat incorrectly mentions "file size in blocks" (should be bytes)
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-8761
> URL: https://issues.apache.org/jira/browse/HADOOP-8761
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Priority: Minor
> Attachments: HADOOP-8761.patch.txt
>
>
> Trivial patch attached corrects the usage information. Stat.java calls FileStatus.getLen(), which is most definitely the file size in bytes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8761) Help for FsShell's Stat incorrectly
mentions "file size in blocks" (should be bytes)
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Philip Zeyliger updated HADOOP-8761:
------------------------------------
Assignee: Philip Zeyliger
Status: Patch Available (was: Open)
Submitting patch for Hudson.
No tests were added.
> Help for FsShell's Stat incorrectly mentions "file size in blocks" (should be bytes)
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-8761
> URL: https://issues.apache.org/jira/browse/HADOOP-8761
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Priority: Minor
> Attachments: HADOOP-8761.patch.txt
>
>
> Trivial patch attached corrects the usage information. Stat.java calls FileStatus.getLen(), which is most definitely the file size in bytes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8761) Help for FsShell's Stat
incorrectly mentions "file size in blocks" (should be bytes)
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447735#comment-13447735 ]
Hadoop QA commented on HADOOP-8761:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12543672/HADOOP-8761.patch.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:
org.apache.hadoop.ha.TestZKFailoverController
org.apache.hadoop.cli.TestCLI
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1398//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1398//console
This message is automatically generated.
> Help for FsShell's Stat incorrectly mentions "file size in blocks" (should be bytes)
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-8761
> URL: https://issues.apache.org/jira/browse/HADOOP-8761
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Priority: Minor
> Attachments: HADOOP-8761.patch.txt
>
>
> Trivial patch attached corrects the usage information. Stat.java calls FileStatus.getLen(), which is most definitely the file size in bytes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8761) Help for FsShell's Stat incorrectly
mentions "file size in blocks" (should be bytes)
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers updated HADOOP-8761:
-----------------------------------
Target Version/s: 2.2.0-alpha
Affects Version/s: 2.0.0-alpha
Targeting for 2.2.0.
Philip, it looks like the TestCLI failure is related, probably because it's expecting some specific help text.
> Help for FsShell's Stat incorrectly mentions "file size in blocks" (should be bytes)
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-8761
> URL: https://issues.apache.org/jira/browse/HADOOP-8761
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Priority: Minor
> Attachments: HADOOP-8761.patch.txt
>
>
> Trivial patch attached corrects the usage information. Stat.java calls FileStatus.getLen(), which is most definitely the file size in bytes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira