You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Hong Tang (JIRA)" <ji...@apache.org> on 2008/09/17 20:55:44 UTC
[jira] Created: (HADOOP-4195) SequenceFile.Writer close() uses
compressor after returning it to CodecPool.
SequenceFile.Writer close() uses compressor after returning it to CodecPool.
----------------------------------------------------------------------------
Key: HADOOP-4195
URL: https://issues.apache.org/jira/browse/HADOOP-4195
Project: Hadoop Core
Issue Type: Bug
Components: io
Affects Versions: 0.18.0
Reporter: Hong Tang
In function SequenceFile.Writer.close() (line 946): The first marked line returns the compressor while the second marked line will use the compressor again. This will lead to a race condition if another thread checks out the compressor between these two marked statements.
{code:title=SequenceFile.java|borderStyle=solid}
public synchronized void close() throws IOException {
CodecPool.returnCompressor(compressor); // <==== compressor returned
keySerializer.close();
uncompressedValSerializer.close();
if (compressedValSerializer != null) {
compressedValSerializer.close(); // <===== compressor used
}
if (out != null) {
// Close the underlying stream iff we own it...
if (ownOutputStream) {
out.close();
} else {
out.flush();
}
out = null;
}
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-4195) SequenceFile.Writer close() uses
compressor after returning it to CodecPool.
Posted by "Jothi Padmanabhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jothi Padmanabhan reassigned HADOOP-4195:
-----------------------------------------
Assignee: Jothi Padmanabhan (was: Arun C Murthy)
> SequenceFile.Writer close() uses compressor after returning it to CodecPool.
> ----------------------------------------------------------------------------
>
> Key: HADOOP-4195
> URL: https://issues.apache.org/jira/browse/HADOOP-4195
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.18.0
> Reporter: Hong Tang
> Assignee: Jothi Padmanabhan
> Fix For: 0.19.0
>
> Attachments: HADOOP-4195_0_20080917.patch
>
>
> In function SequenceFile.Writer.close() (line 946): The first marked line returns the compressor while the second marked line will use the compressor again. This will lead to a race condition if another thread checks out the compressor between these two marked statements.
> {code:title=SequenceFile.java|borderStyle=solid}
> public synchronized void close() throws IOException {
> CodecPool.returnCompressor(compressor); // <==== compressor returned
>
> keySerializer.close();
> uncompressedValSerializer.close();
> if (compressedValSerializer != null) {
> compressedValSerializer.close(); // <===== compressor used
> }
> if (out != null) {
>
> // Close the underlying stream iff we own it...
> if (ownOutputStream) {
> out.close();
> } else {
> out.flush();
> }
> out = null;
> }
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4195) SequenceFile.Writer close() uses
compressor after returning it to CodecPool.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631950#action_12631950 ]
Hong Tang commented on HADOOP-4195:
-----------------------------------
Patch verified to work.
> SequenceFile.Writer close() uses compressor after returning it to CodecPool.
> ----------------------------------------------------------------------------
>
> Key: HADOOP-4195
> URL: https://issues.apache.org/jira/browse/HADOOP-4195
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.18.0
> Reporter: Hong Tang
> Assignee: Arun C Murthy
> Fix For: 0.19.0
>
> Attachments: HADOOP-4195_0_20080917.patch
>
>
> In function SequenceFile.Writer.close() (line 946): The first marked line returns the compressor while the second marked line will use the compressor again. This will lead to a race condition if another thread checks out the compressor between these two marked statements.
> {code:title=SequenceFile.java|borderStyle=solid}
> public synchronized void close() throws IOException {
> CodecPool.returnCompressor(compressor); // <==== compressor returned
>
> keySerializer.close();
> uncompressedValSerializer.close();
> if (compressedValSerializer != null) {
> compressedValSerializer.close(); // <===== compressor used
> }
> if (out != null) {
>
> // Close the underlying stream iff we own it...
> if (ownOutputStream) {
> out.close();
> } else {
> out.flush();
> }
> out = null;
> }
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4195) SequenceFile.Writer close() uses
compressor after returning it to CodecPool.
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-4195:
----------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Arun!
> SequenceFile.Writer close() uses compressor after returning it to CodecPool.
> ----------------------------------------------------------------------------
>
> Key: HADOOP-4195
> URL: https://issues.apache.org/jira/browse/HADOOP-4195
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.18.0
> Reporter: Hong Tang
> Assignee: Arun C Murthy
> Fix For: 0.19.0
>
> Attachments: HADOOP-4195_0_20080917.patch
>
>
> In function SequenceFile.Writer.close() (line 946): The first marked line returns the compressor while the second marked line will use the compressor again. This will lead to a race condition if another thread checks out the compressor between these two marked statements.
> {code:title=SequenceFile.java|borderStyle=solid}
> public synchronized void close() throws IOException {
> CodecPool.returnCompressor(compressor); // <==== compressor returned
>
> keySerializer.close();
> uncompressedValSerializer.close();
> if (compressedValSerializer != null) {
> compressedValSerializer.close(); // <===== compressor used
> }
> if (out != null) {
>
> // Close the underlying stream iff we own it...
> if (ownOutputStream) {
> out.close();
> } else {
> out.flush();
> }
> out = null;
> }
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4195) SequenceFile.Writer close() uses
compressor after returning it to CodecPool.
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-4195:
----------------------------------
Fix Version/s: 0.19.0
Assignee: Arun C Murthy
> SequenceFile.Writer close() uses compressor after returning it to CodecPool.
> ----------------------------------------------------------------------------
>
> Key: HADOOP-4195
> URL: https://issues.apache.org/jira/browse/HADOOP-4195
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.18.0
> Reporter: Hong Tang
> Assignee: Arun C Murthy
> Fix For: 0.19.0
>
>
> In function SequenceFile.Writer.close() (line 946): The first marked line returns the compressor while the second marked line will use the compressor again. This will lead to a race condition if another thread checks out the compressor between these two marked statements.
> {code:title=SequenceFile.java|borderStyle=solid}
> public synchronized void close() throws IOException {
> CodecPool.returnCompressor(compressor); // <==== compressor returned
>
> keySerializer.close();
> uncompressedValSerializer.close();
> if (compressedValSerializer != null) {
> compressedValSerializer.close(); // <===== compressor used
> }
> if (out != null) {
>
> // Close the underlying stream iff we own it...
> if (ownOutputStream) {
> out.close();
> } else {
> out.flush();
> }
> out = null;
> }
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4195) SequenceFile.Writer close() uses
compressor after returning it to CodecPool.
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-4195:
----------------------------------
Attachment: HADOOP-4195_0_20080917.patch
Straight-forward fix... can you pls verify it Hong? Thanks!
> SequenceFile.Writer close() uses compressor after returning it to CodecPool.
> ----------------------------------------------------------------------------
>
> Key: HADOOP-4195
> URL: https://issues.apache.org/jira/browse/HADOOP-4195
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.18.0
> Reporter: Hong Tang
> Assignee: Arun C Murthy
> Fix For: 0.19.0
>
> Attachments: HADOOP-4195_0_20080917.patch
>
>
> In function SequenceFile.Writer.close() (line 946): The first marked line returns the compressor while the second marked line will use the compressor again. This will lead to a race condition if another thread checks out the compressor between these two marked statements.
> {code:title=SequenceFile.java|borderStyle=solid}
> public synchronized void close() throws IOException {
> CodecPool.returnCompressor(compressor); // <==== compressor returned
>
> keySerializer.close();
> uncompressedValSerializer.close();
> if (compressedValSerializer != null) {
> compressedValSerializer.close(); // <===== compressor used
> }
> if (out != null) {
>
> // Close the underlying stream iff we own it...
> if (ownOutputStream) {
> out.close();
> } else {
> out.flush();
> }
> out = null;
> }
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-4195) SequenceFile.Writer close() uses
compressor after returning it to CodecPool.
Posted by "Jothi Padmanabhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jothi Padmanabhan reassigned HADOOP-4195:
-----------------------------------------
Assignee: Arun C Murthy (was: Jothi Padmanabhan)
Reassigning, assigned the wrong patch
> SequenceFile.Writer close() uses compressor after returning it to CodecPool.
> ----------------------------------------------------------------------------
>
> Key: HADOOP-4195
> URL: https://issues.apache.org/jira/browse/HADOOP-4195
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.18.0
> Reporter: Hong Tang
> Assignee: Arun C Murthy
> Fix For: 0.19.0
>
> Attachments: HADOOP-4195_0_20080917.patch
>
>
> In function SequenceFile.Writer.close() (line 946): The first marked line returns the compressor while the second marked line will use the compressor again. This will lead to a race condition if another thread checks out the compressor between these two marked statements.
> {code:title=SequenceFile.java|borderStyle=solid}
> public synchronized void close() throws IOException {
> CodecPool.returnCompressor(compressor); // <==== compressor returned
>
> keySerializer.close();
> uncompressedValSerializer.close();
> if (compressedValSerializer != null) {
> compressedValSerializer.close(); // <===== compressor used
> }
> if (out != null) {
>
> // Close the underlying stream iff we own it...
> if (ownOutputStream) {
> out.close();
> } else {
> out.flush();
> }
> out = null;
> }
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4195) SequenceFile.Writer close() uses
compressor after returning it to CodecPool.
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-4195:
----------------------------------
Status: Patch Available (was: Open)
> SequenceFile.Writer close() uses compressor after returning it to CodecPool.
> ----------------------------------------------------------------------------
>
> Key: HADOOP-4195
> URL: https://issues.apache.org/jira/browse/HADOOP-4195
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.18.0
> Reporter: Hong Tang
> Assignee: Arun C Murthy
> Fix For: 0.19.0
>
> Attachments: HADOOP-4195_0_20080917.patch
>
>
> In function SequenceFile.Writer.close() (line 946): The first marked line returns the compressor while the second marked line will use the compressor again. This will lead to a race condition if another thread checks out the compressor between these two marked statements.
> {code:title=SequenceFile.java|borderStyle=solid}
> public synchronized void close() throws IOException {
> CodecPool.returnCompressor(compressor); // <==== compressor returned
>
> keySerializer.close();
> uncompressedValSerializer.close();
> if (compressedValSerializer != null) {
> compressedValSerializer.close(); // <===== compressor used
> }
> if (out != null) {
>
> // Close the underlying stream iff we own it...
> if (ownOutputStream) {
> out.close();
> } else {
> out.flush();
> }
> out = null;
> }
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4195) SequenceFile.Writer close() uses
compressor after returning it to CodecPool.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12633323#action_12633323 ]
Hudson commented on HADOOP-4195:
--------------------------------
Integrated in Hadoop-trunk #611 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/611/])
> SequenceFile.Writer close() uses compressor after returning it to CodecPool.
> ----------------------------------------------------------------------------
>
> Key: HADOOP-4195
> URL: https://issues.apache.org/jira/browse/HADOOP-4195
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.18.0
> Reporter: Hong Tang
> Assignee: Arun C Murthy
> Fix For: 0.19.0
>
> Attachments: HADOOP-4195_0_20080917.patch
>
>
> In function SequenceFile.Writer.close() (line 946): The first marked line returns the compressor while the second marked line will use the compressor again. This will lead to a race condition if another thread checks out the compressor between these two marked statements.
> {code:title=SequenceFile.java|borderStyle=solid}
> public synchronized void close() throws IOException {
> CodecPool.returnCompressor(compressor); // <==== compressor returned
>
> keySerializer.close();
> uncompressedValSerializer.close();
> if (compressedValSerializer != null) {
> compressedValSerializer.close(); // <===== compressor used
> }
> if (out != null) {
>
> // Close the underlying stream iff we own it...
> if (ownOutputStream) {
> out.close();
> } else {
> out.flush();
> }
> out = null;
> }
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.