You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Raghu Angadi (JIRA)" <ji...@apache.org> on 2008/02/20 21:37:44 UTC
[jira] Created: (HADOOP-2863) FSDataOutputStream should not flush()
inside close().
FSDataOutputStream should not flush() inside close().
-----------------------------------------------------
Key: HADOOP-2863
URL: https://issues.apache.org/jira/browse/HADOOP-2863
Project: Hadoop Core
Issue Type: Bug
Components: fs
Reporter: Raghu Angadi
Fix For: 0.17.0
Why does FSDataOutputStream.close() call flush()? This stream itself does not store any data that it needs to flush. It is a wrapper and it should just invoke its outputstream's close().
For. e.g one bad side effect is that, in the case of DFSOutputStream which extends FSOutputSummer, flush() inside close sends the current data even though FSOutputSummer might have some data.. this left over data will be sent in side close() (so it sends data in two different packets instead of one). Other filesystems might have similar side effects.
I will submit a patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2863) FSDataOutputStream should not flush()
inside close().
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-2863:
---------------------------------
Attachment: HADOOP-2863.patch
The updated patch just removes flush() inside close() for FSDataOutputStream().
> FSDataOutputStream should not flush() inside close().
> -----------------------------------------------------
>
> Key: HADOOP-2863
> URL: https://issues.apache.org/jira/browse/HADOOP-2863
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.17.0
>
> Attachments: HADOOP-2863.patch, HADOOP-2863.patch
>
>
> Why does FSDataOutputStream.close() call flush()? This stream itself does not store any data that it needs to flush. It is a wrapper and it should just invoke its outputstream's close().
> For. e.g one bad side effect is that, in the case of DFSOutputStream which extends FSOutputSummer, flush() inside close sends the current data even though FSOutputSummer might have some data.. this left over data will be sent in side close() (so it sends data in two different packets instead of one). Other filesystems might have similar side effects.
> I will submit a patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-2863) FSDataOutputStream should not
flush() inside close().
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi reassigned HADOOP-2863:
------------------------------------
Assignee: Raghu Angadi
> FSDataOutputStream should not flush() inside close().
> -----------------------------------------------------
>
> Key: HADOOP-2863
> URL: https://issues.apache.org/jira/browse/HADOOP-2863
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.17.0
>
> Attachments: HADOOP-2863.patch
>
>
> Why does FSDataOutputStream.close() call flush()? This stream itself does not store any data that it needs to flush. It is a wrapper and it should just invoke its outputstream's close().
> For. e.g one bad side effect is that, in the case of DFSOutputStream which extends FSOutputSummer, flush() inside close sends the current data even though FSOutputSummer might have some data.. this left over data will be sent in side close() (so it sends data in two different packets instead of one). Other filesystems might have similar side effects.
> I will submit a patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2863) FSDataOutputStream should not flush()
inside close().
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-2863:
---------------------------------
Description:
Why does FSDataOutputStream.close() call flush()? This stream itself does not store any data that it needs to flush. It is a wrapper and it should just invoke its outputstream's close().
For. e.g one bad side effect is that, in the case of DFSOutputStream which extends FSOutputSummer, flush() inside close sends the current data even though FSOutputSummer might have some data.. this left over data will be sent in side close() (so it sends data in two different packets instead of one). Other filesystems might have similar side effects.
I will submit a patch.
was:
Why does FSDataOutputStream.close() call flush()? This stream itself does not store any data that it needs to flush. It is a wrapper and it should just invoke its outputstream's close().
For. e.g one bad side effect is that, in the case of DFSOutputStream which extends FSOutputSummer, flush() inside close sends the current data even though FSOutputSummer might have some data.. this left over data will be sent in side close() (so it sends data in two different packets instead of one). Other filesystems might have similar side effects.
I will submit a patch.
Fix Version/s: (was: 0.17.0)
bq. For. e.g one bad side effect is that, in the case of DFSOutputStream which extends FSOutputSummer, flush() inside close sends the current data even though FSOutputSummer might have some data.. this left over data will be sent in side close() (so it sends data in two different packets instead of one). Other filesystems might have similar side effects.
I am fixing the above as part of HADOOP-1702 by renaming {{DFSOutputStream.flush()}} to {{flushInternal()}} (because it is not a normal flush() and needs to be called only inside close()).
> FSDataOutputStream should not flush() inside close().
> -----------------------------------------------------
>
> Key: HADOOP-2863
> URL: https://issues.apache.org/jira/browse/HADOOP-2863
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Attachments: HADOOP-2863.patch, HADOOP-2863.patch
>
>
> Why does FSDataOutputStream.close() call flush()? This stream itself does not store any data that it needs to flush. It is a wrapper and it should just invoke its outputstream's close().
> For. e.g one bad side effect is that, in the case of DFSOutputStream which extends FSOutputSummer, flush() inside close sends the current data even though FSOutputSummer might have some data.. this left over data will be sent in side close() (so it sends data in two different packets instead of one). Other filesystems might have similar side effects.
> I will submit a patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-2863) FSDataOutputStream should not
flush() inside close().
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi resolved HADOOP-2863.
----------------------------------
Resolution: Fixed
Fix Version/s: 0.17.0
Is part of HADOOP-2657.
> FSDataOutputStream should not flush() inside close().
> -----------------------------------------------------
>
> Key: HADOOP-2863
> URL: https://issues.apache.org/jira/browse/HADOOP-2863
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.17.0
>
> Attachments: HADOOP-2863.patch, HADOOP-2863.patch
>
>
> Why does FSDataOutputStream.close() call flush()? This stream itself does not store any data that it needs to flush. It is a wrapper and it should just invoke its outputstream's close().
> For. e.g one bad side effect is that, in the case of DFSOutputStream which extends FSOutputSummer, flush() inside close sends the current data even though FSOutputSummer might have some data.. this left over data will be sent in side close() (so it sends data in two different packets instead of one). Other filesystems might have similar side effects.
> I will submit a patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2863) FSDataOutputStream should not flush()
inside close().
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-2863:
---------------------------------
Attachment: HADOOP-2863.patch
It removes close() from FSDataOutputStream and FSDataOutputStream$PositionCache since it is not required.
> FSDataOutputStream should not flush() inside close().
> -----------------------------------------------------
>
> Key: HADOOP-2863
> URL: https://issues.apache.org/jira/browse/HADOOP-2863
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Raghu Angadi
> Fix For: 0.17.0
>
> Attachments: HADOOP-2863.patch
>
>
> Why does FSDataOutputStream.close() call flush()? This stream itself does not store any data that it needs to flush. It is a wrapper and it should just invoke its outputstream's close().
> For. e.g one bad side effect is that, in the case of DFSOutputStream which extends FSOutputSummer, flush() inside close sends the current data even though FSOutputSummer might have some data.. this left over data will be sent in side close() (so it sends data in two different packets instead of one). Other filesystems might have similar side effects.
> I will submit a patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2863) FSDataOutputStream should not
flush() inside close().
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571135#action_12571135 ]
dhruba borthakur commented on HADOOP-2863:
------------------------------------------
+1 for the latest version of this patch.
> FSDataOutputStream should not flush() inside close().
> -----------------------------------------------------
>
> Key: HADOOP-2863
> URL: https://issues.apache.org/jira/browse/HADOOP-2863
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Attachments: HADOOP-2863.patch, HADOOP-2863.patch
>
>
> Why does FSDataOutputStream.close() call flush()? This stream itself does not store any data that it needs to flush. It is a wrapper and it should just invoke its outputstream's close().
> For. e.g one bad side effect is that, in the case of DFSOutputStream which extends FSOutputSummer, flush() inside close sends the current data even though FSOutputSummer might have some data.. this left over data will be sent in side close() (so it sends data in two different packets instead of one). Other filesystems might have similar side effects.
> I will submit a patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.