You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Kristofer Tomasette (Created) (JIRA)" <ji...@apache.org> on 2011/11/11 18:49:53 UTC
[jira] [Created] (HADOOP-7817) RawLocalFileSystem.append() should
give FSDataOutputStream with accurate .getPos()
RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos()
----------------------------------------------------------------------------------
Key: HADOOP-7817
URL: https://issues.apache.org/jira/browse/HADOOP-7817
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 0.24.0
Reporter: Kristofer Tomasette
Priority: Minor
When RawLocalFileSyste.append() is called it returns an FSDataOutputStream whose .getPos() returns 0.
getPos() should return position in the file where appends will start writing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7817) RawLocalFileSystem.append() should
give FSDataOutputStream with accurate .getPos()
Posted by "Kristofer Tomasette (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristofer Tomasette updated HADOOP-7817:
----------------------------------------
Affects Version/s: (was: 0.24.0)
1.0.0
> RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos()
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-7817
> URL: https://issues.apache.org/jira/browse/HADOOP-7817
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 1.0.0
> Reporter: Kristofer Tomasette
> Priority: Minor
> Attachments: HADOOP-7817.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When RawLocalFileSyste.append() is called it returns an FSDataOutputStream whose .getPos() returns 0.
> getPos() should return position in the file where appends will start writing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7817) RawLocalFileSystem.append() should
give FSDataOutputStream with accurate .getPos()
Posted by "Kristofer Tomasette (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148668#comment-13148668 ]
Kristofer Tomasette commented on HADOOP-7817:
---------------------------------------------
FWIW:
DFSClient.append does:
return new FSDataOutputStream(out, statistics, out.getInitialLen());
Was planning on submitting a patch on RawLocalFileSystem w/
public FSDataOutputStream append(Path f, int bufferSize,
Progressable progress) throws IOException {
if (!exists(f)) {
throw new FileNotFoundException("File " + f + " not found");
}
FileStatus status = getFileStatus(f);
if (status.isDirectory()) {
throw new IOException("Cannot append to a diretory (=" + f + " )");
}
return new FSDataOutputStream(new BufferedOutputStream(
new LocalFSFileOutputStream(f, true), bufferSize), statistics,status.getLen());
}
> RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos()
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-7817
> URL: https://issues.apache.org/jira/browse/HADOOP-7817
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.24.0
> Reporter: Kristofer Tomasette
> Priority: Minor
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When RawLocalFileSyste.append() is called it returns an FSDataOutputStream whose .getPos() returns 0.
> getPos() should return position in the file where appends will start writing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7817) RawLocalFileSystem.append() should
give FSDataOutputStream with accurate .getPos()
Posted by "Kristofer Tomasette (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristofer Tomasette updated HADOOP-7817:
----------------------------------------
Attachment: HADOOP-7817.patch
> RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos()
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-7817
> URL: https://issues.apache.org/jira/browse/HADOOP-7817
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.24.0
> Reporter: Kristofer Tomasette
> Priority: Minor
> Attachments: HADOOP-7817.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When RawLocalFileSyste.append() is called it returns an FSDataOutputStream whose .getPos() returns 0.
> getPos() should return position in the file where appends will start writing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7817) RawLocalFileSystem.append() should
give FSDataOutputStream with accurate .getPos()
Posted by "Kristofer Tomasette (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148612#comment-13148612 ]
Kristofer Tomasette commented on HADOOP-7817:
---------------------------------------------
patch forthcoming
> RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos()
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-7817
> URL: https://issues.apache.org/jira/browse/HADOOP-7817
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.24.0
> Reporter: Kristofer Tomasette
> Priority: Minor
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When RawLocalFileSyste.append() is called it returns an FSDataOutputStream whose .getPos() returns 0.
> getPos() should return position in the file where appends will start writing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7817) RawLocalFileSystem.append() should
give FSDataOutputStream with accurate .getPos()
Posted by "Kristofer Tomasette (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179039#comment-13179039 ]
Kristofer Tomasette commented on HADOOP-7817:
---------------------------------------------
Is anyone looking at this? I have another patch that depends on this and would like to get some feedback.
Thanks!
> RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos()
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-7817
> URL: https://issues.apache.org/jira/browse/HADOOP-7817
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.24.0
> Reporter: Kristofer Tomasette
> Priority: Minor
> Attachments: HADOOP-7817.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When RawLocalFileSyste.append() is called it returns an FSDataOutputStream whose .getPos() returns 0.
> getPos() should return position in the file where appends will start writing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7817) RawLocalFileSystem.append() should
give FSDataOutputStream with accurate .getPos()
Posted by "Kristofer Tomasette (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristofer Tomasette updated HADOOP-7817:
----------------------------------------
Status: Patch Available (was: Open)
> RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos()
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-7817
> URL: https://issues.apache.org/jira/browse/HADOOP-7817
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.24.0
> Reporter: Kristofer Tomasette
> Priority: Minor
> Attachments: HADOOP-7817.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When RawLocalFileSyste.append() is called it returns an FSDataOutputStream whose .getPos() returns 0.
> getPos() should return position in the file where appends will start writing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7817) RawLocalFileSystem.append() should
give FSDataOutputStream with accurate .getPos()
Posted by "Kristofer Tomasette (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164016#comment-13164016 ]
Kristofer Tomasette commented on HADOOP-7817:
---------------------------------------------
I think the -1 javadoc mentioned above is not related to my patch.
> RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos()
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-7817
> URL: https://issues.apache.org/jira/browse/HADOOP-7817
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.24.0
> Reporter: Kristofer Tomasette
> Priority: Minor
> Attachments: HADOOP-7817.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When RawLocalFileSyste.append() is called it returns an FSDataOutputStream whose .getPos() returns 0.
> getPos() should return position in the file where appends will start writing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7817) RawLocalFileSystem.append() should
give FSDataOutputStream with accurate .getPos()
Posted by "Uma Maheswara Rao G (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148650#comment-13148650 ]
Uma Maheswara Rao G commented on HADOOP-7817:
---------------------------------------------
Yes, the position will come as zero, because it directly opens the FileOutPutstream in append mode. This will ensures that the data to go at the end of the file.
fixing this would be simple, we can set the position on fileChannel position to the length of the file. But i am not sure this is really a bug or not, because the author intention is to leave as it is equal behavior with FileOutPutStream when opening in append mode....?
> RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos()
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-7817
> URL: https://issues.apache.org/jira/browse/HADOOP-7817
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.24.0
> Reporter: Kristofer Tomasette
> Priority: Minor
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When RawLocalFileSyste.append() is called it returns an FSDataOutputStream whose .getPos() returns 0.
> getPos() should return position in the file where appends will start writing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7817) RawLocalFileSystem.append() should
give FSDataOutputStream with accurate .getPos()
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163683#comment-13163683 ]
Hadoop QA commented on HADOOP-7817:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12506157/HADOOP-7817.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 9 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/437//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/437//console
This message is automatically generated.
> RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos()
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-7817
> URL: https://issues.apache.org/jira/browse/HADOOP-7817
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.24.0
> Reporter: Kristofer Tomasette
> Priority: Minor
> Attachments: HADOOP-7817.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When RawLocalFileSyste.append() is called it returns an FSDataOutputStream whose .getPos() returns 0.
> getPos() should return position in the file where appends will start writing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira