You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Devaraj Das (JIRA)" <ji...@apache.org> on 2008/03/17 07:47:24 UTC
[jira] Created: (HADOOP-3030)
InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while
reserving space for the file in question
InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while reserving space for the file in question
--------------------------------------------------------------------------------------------------------------------
Key: HADOOP-3030
URL: https://issues.apache.org/jira/browse/HADOOP-3030
Project: Hadoop Core
Issue Type: Bug
Components: fs
Reporter: Devaraj Das
Fix For: 0.17.0
The return statement code in InMemoryFileSystem.reserveSpaceWithCheckSum looks like
{noformat}
return (mfs.reserveSpace(f, size) && mfs.reserveSpace(getChecksumFile(f), checksumSize));
{noformat}
This should be broken up to check for successful reserveSpace for each of the components. In some cases, we might reserve space for the first component and fail while doing the same for the second (checksum file). This will lead to wastage of space since we don't un-reserve the space we got for the first component. This usually won't happen due to the policy associated with creating a file in the InMemoryFileSystem (since the checksum component is usually very small) but still it should be fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3030)
InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while
reserving space for the file in question
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-3030:
--------------------------------
Status: Patch Available (was: Open)
> InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while reserving space for the file in question
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3030
> URL: https://issues.apache.org/jira/browse/HADOOP-3030
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.17.0
>
> Attachments: 3030.patch, hadoop-3030.patch
>
>
> The return statement code in InMemoryFileSystem.reserveSpaceWithCheckSum looks like
> {noformat}
> return (mfs.reserveSpace(f, size) && mfs.reserveSpace(getChecksumFile(f), checksumSize));
> {noformat}
> This should be broken up to check for successful reserveSpace for each of the components. In some cases, we might reserve space for the first component and fail while doing the same for the second (checksum file). This will lead to wastage of space since we don't un-reserve the space we got for the first component. This usually won't happen due to the policy associated with creating a file in the InMemoryFileSystem (since the checksum component is usually very small) but still it should be fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-3030)
InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while
reserving space for the file in question
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das reassigned HADOOP-3030:
-----------------------------------
Assignee: Devaraj Das
> InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while reserving space for the file in question
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3030
> URL: https://issues.apache.org/jira/browse/HADOOP-3030
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.17.0
>
> Attachments: hadoop-3030.patch
>
>
> The return statement code in InMemoryFileSystem.reserveSpaceWithCheckSum looks like
> {noformat}
> return (mfs.reserveSpace(f, size) && mfs.reserveSpace(getChecksumFile(f), checksumSize));
> {noformat}
> This should be broken up to check for successful reserveSpace for each of the components. In some cases, we might reserve space for the first component and fail while doing the same for the second (checksum file). This will lead to wastage of space since we don't un-reserve the space we got for the first component. This usually won't happen due to the policy associated with creating a file in the InMemoryFileSystem (since the checksum component is usually very small) but still it should be fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3030)
InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while
reserving space for the file in question
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579459#action_12579459 ]
Hadoop QA commented on HADOOP-3030:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12378021/hadoop-3030.patch
against trunk revision 619744.
@author +1. The patch does not contain any @author tags.
tests included -1. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests -1. The patch failed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1982/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1982/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1982/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1982/console
This message is automatically generated.
> InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while reserving space for the file in question
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3030
> URL: https://issues.apache.org/jira/browse/HADOOP-3030
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.17.0
>
> Attachments: hadoop-3030.patch
>
>
> The return statement code in InMemoryFileSystem.reserveSpaceWithCheckSum looks like
> {noformat}
> return (mfs.reserveSpace(f, size) && mfs.reserveSpace(getChecksumFile(f), checksumSize));
> {noformat}
> This should be broken up to check for successful reserveSpace for each of the components. In some cases, we might reserve space for the first component and fail while doing the same for the second (checksum file). This will lead to wastage of space since we don't un-reserve the space we got for the first component. This usually won't happen due to the policy associated with creating a file in the InMemoryFileSystem (since the checksum component is usually very small) but still it should be fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3030)
InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while
reserving space for the file in question
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580446#action_12580446 ]
Hadoop QA commented on HADOOP-3030:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12378099/3030.patch
against trunk revision 619744.
@author +1. The patch does not contain any @author tags.
tests included -1. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1997/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1997/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1997/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1997/console
This message is automatically generated.
> InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while reserving space for the file in question
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3030
> URL: https://issues.apache.org/jira/browse/HADOOP-3030
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.17.0
>
> Attachments: 3030.patch, hadoop-3030.patch
>
>
> The return statement code in InMemoryFileSystem.reserveSpaceWithCheckSum looks like
> {noformat}
> return (mfs.reserveSpace(f, size) && mfs.reserveSpace(getChecksumFile(f), checksumSize));
> {noformat}
> This should be broken up to check for successful reserveSpace for each of the components. In some cases, we might reserve space for the first component and fail while doing the same for the second (checksum file). This will lead to wastage of space since we don't un-reserve the space we got for the first component. This usually won't happen due to the policy associated with creating a file in the InMemoryFileSystem (since the checksum component is usually very small) but still it should be fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3030)
InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while
reserving space for the file in question
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-3030:
--------------------------------
Status: Open (was: Patch Available)
> InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while reserving space for the file in question
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3030
> URL: https://issues.apache.org/jira/browse/HADOOP-3030
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.17.0
>
> Attachments: 3030.patch, hadoop-3030.patch
>
>
> The return statement code in InMemoryFileSystem.reserveSpaceWithCheckSum looks like
> {noformat}
> return (mfs.reserveSpace(f, size) && mfs.reserveSpace(getChecksumFile(f), checksumSize));
> {noformat}
> This should be broken up to check for successful reserveSpace for each of the components. In some cases, we might reserve space for the first component and fail while doing the same for the second (checksum file). This will lead to wastage of space since we don't un-reserve the space we got for the first component. This usually won't happen due to the policy associated with creating a file in the InMemoryFileSystem (since the checksum component is usually very small) but still it should be fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3030)
InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while
reserving space for the file in question
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-3030:
--------------------------------
Attachment: 3030.patch
This patch addresses Chris's comment. Thanks Chris!
> InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while reserving space for the file in question
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3030
> URL: https://issues.apache.org/jira/browse/HADOOP-3030
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.17.0
>
> Attachments: 3030.patch, hadoop-3030.patch
>
>
> The return statement code in InMemoryFileSystem.reserveSpaceWithCheckSum looks like
> {noformat}
> return (mfs.reserveSpace(f, size) && mfs.reserveSpace(getChecksumFile(f), checksumSize));
> {noformat}
> This should be broken up to check for successful reserveSpace for each of the components. In some cases, we might reserve space for the first component and fail while doing the same for the second (checksum file). This will lead to wastage of space since we don't un-reserve the space we got for the first component. This usually won't happen due to the policy associated with creating a file in the InMemoryFileSystem (since the checksum component is usually very small) but still it should be fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3030)
InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while
reserving space for the file in question
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-3030:
--------------------------------
Attachment: hadoop-3030.patch
Attached patch breaks up the reserveSpace calls to two and does an unreserve, if error is encountered for the checksum file space reservation.
> InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while reserving space for the file in question
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3030
> URL: https://issues.apache.org/jira/browse/HADOOP-3030
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Devaraj Das
> Fix For: 0.17.0
>
> Attachments: hadoop-3030.patch
>
>
> The return statement code in InMemoryFileSystem.reserveSpaceWithCheckSum looks like
> {noformat}
> return (mfs.reserveSpace(f, size) && mfs.reserveSpace(getChecksumFile(f), checksumSize));
> {noformat}
> This should be broken up to check for successful reserveSpace for each of the components. In some cases, we might reserve space for the first component and fail while doing the same for the second (checksum file). This will lead to wastage of space since we don't un-reserve the space we got for the first component. This usually won't happen due to the policy associated with creating a file in the InMemoryFileSystem (since the checksum component is usually very small) but still it should be fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3030)
InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while
reserving space for the file in question
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-3030:
--------------------------------
Status: Patch Available (was: Open)
> InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while reserving space for the file in question
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3030
> URL: https://issues.apache.org/jira/browse/HADOOP-3030
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.17.0
>
> Attachments: hadoop-3030.patch
>
>
> The return statement code in InMemoryFileSystem.reserveSpaceWithCheckSum looks like
> {noformat}
> return (mfs.reserveSpace(f, size) && mfs.reserveSpace(getChecksumFile(f), checksumSize));
> {noformat}
> This should be broken up to check for successful reserveSpace for each of the components. In some cases, we might reserve space for the first component and fail while doing the same for the second (checksum file). This will lead to wastage of space since we don't un-reserve the space we got for the first component. This usually won't happen due to the policy associated with creating a file in the InMemoryFileSystem (since the checksum component is usually very small) but still it should be fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3030)
InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while
reserving space for the file in question
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580745#action_12580745 ]
Hudson commented on HADOOP-3030:
--------------------------------
Integrated in Hadoop-trunk #434 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/434/])
> InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while reserving space for the file in question
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3030
> URL: https://issues.apache.org/jira/browse/HADOOP-3030
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.17.0
>
> Attachments: 3030.patch, hadoop-3030.patch
>
>
> The return statement code in InMemoryFileSystem.reserveSpaceWithCheckSum looks like
> {noformat}
> return (mfs.reserveSpace(f, size) && mfs.reserveSpace(getChecksumFile(f), checksumSize));
> {noformat}
> This should be broken up to check for successful reserveSpace for each of the components. In some cases, we might reserve space for the first component and fail while doing the same for the second (checksum file). This will lead to wastage of space since we don't un-reserve the space we got for the first component. This usually won't happen due to the policy associated with creating a file in the InMemoryFileSystem (since the checksum component is usually very small) but still it should be fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3030)
InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while
reserving space for the file in question
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579696#action_12579696 ]
Chris Douglas commented on HADOOP-3030:
---------------------------------------
unreserveSpace should probably follow the locking semantics observed by reserveSpace, though you're already managing it explicitly, it's a private inner class, etc. etc. Most other methods on RawInMemoryFileSystem observe that pattern, so for consistency it might be worthwhile. Otherwise +1
> InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while reserving space for the file in question
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3030
> URL: https://issues.apache.org/jira/browse/HADOOP-3030
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.17.0
>
> Attachments: hadoop-3030.patch
>
>
> The return statement code in InMemoryFileSystem.reserveSpaceWithCheckSum looks like
> {noformat}
> return (mfs.reserveSpace(f, size) && mfs.reserveSpace(getChecksumFile(f), checksumSize));
> {noformat}
> This should be broken up to check for successful reserveSpace for each of the components. In some cases, we might reserve space for the first component and fail while doing the same for the second (checksum file). This will lead to wastage of space since we don't un-reserve the space we got for the first component. This usually won't happen due to the policy associated with creating a file in the InMemoryFileSystem (since the checksum component is usually very small) but still it should be fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3030)
InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while
reserving space for the file in question
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-3030:
----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Devaraj!
> InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while reserving space for the file in question
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3030
> URL: https://issues.apache.org/jira/browse/HADOOP-3030
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.17.0
>
> Attachments: 3030.patch, hadoop-3030.patch
>
>
> The return statement code in InMemoryFileSystem.reserveSpaceWithCheckSum looks like
> {noformat}
> return (mfs.reserveSpace(f, size) && mfs.reserveSpace(getChecksumFile(f), checksumSize));
> {noformat}
> This should be broken up to check for successful reserveSpace for each of the components. In some cases, we might reserve space for the first component and fail while doing the same for the second (checksum file). This will lead to wastage of space since we don't un-reserve the space we got for the first component. This usually won't happen due to the policy associated with creating a file in the InMemoryFileSystem (since the checksum component is usually very small) but still it should be fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.