You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Eli Collins (JIRA)" <ji...@apache.org> on 2010/06/03 01:11:56 UTC
[jira] Created: (HADOOP-6803) ZlibCompressor hangs on close
ZlibCompressor hangs on close
-----------------------------
Key: HADOOP-6803
URL: https://issues.apache.org/jira/browse/HADOOP-6803
Project: Hadoop Common
Issue Type: Bug
Components: io
Affects Versions: 0.21.0, 0.22.0
Reporter: Eli Collins
Assignee: Eli Collins
Fix For: 0.21.0, 0.22.0
Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
{code}
while (!compressor.finished()) {
compress();
}
{code}
I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6803) ZlibCompressor hangs on close
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-6803:
--------------------------------
Status: Patch Available (was: Open)
I confirmed the test passes locally with the native lib built. On hudson the native test should warn and bail if the native library is not built when the test is run.
> ZlibCompressor hangs on close
> -----------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.21.0, 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.21.0, 0.22.0
>
> Attachments: hadoop-6803-1.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6803) Add native gzip read/write coverage
to TestCodec
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-6803:
--------------------------------
Status: Patch Available (was: Open)
> Add native gzip read/write coverage to TestCodec
> -------------------------------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Test
> Components: io
> Affects Versions: 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.22.0
>
> Attachments: hadoop-6803-1.patch, hadoop-6803-2.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6803) Add native gzip read/write coverage
to TestCodec
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-6803:
--------------------------------
Summary: Add native gzip read/write coverage to TestCodec (was: ZlibCompressor hangs on close)
Issue Type: Test (was: Bug)
> Add native gzip read/write coverage to TestCodec
> -------------------------------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Test
> Components: io
> Affects Versions: 0.21.0, 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.21.0, 0.22.0
>
> Attachments: hadoop-6803-1.patch, hadoop-6803-2.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6803) Add native gzip read/write coverage
to TestCodec
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-6803:
--------------------------------
Status: Open (was: Patch Available)
Affects Version/s: (was: 0.21.0)
Fix Version/s: (was: 0.21.0)
> Add native gzip read/write coverage to TestCodec
> -------------------------------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Test
> Components: io
> Affects Versions: 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.22.0
>
> Attachments: hadoop-6803-1.patch, hadoop-6803-2.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6803) ZlibCompressor hangs on close
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-6803:
--------------------------------
Attachment: hadoop-6803-2.patch
Attached patch. Reverts the ZlibCompressor change, keeps the test. Thanks Chris.
> ZlibCompressor hangs on close
> -----------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.21.0, 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.21.0, 0.22.0
>
> Attachments: hadoop-6803-1.patch, hadoop-6803-2.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6803) Add native gzip read/write coverage
to TestCodec
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-6803:
--------------------------------
Status: Open (was: Patch Available)
> Add native gzip read/write coverage to TestCodec
> -------------------------------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Test
> Components: io
> Affects Versions: 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.22.0
>
> Attachments: hadoop-6803-1.patch, hadoop-6803-2.patch, hadoop-6803-3.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6803) Add native gzip read/write coverage
to TestCodec
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899630#action_12899630 ]
Tom White commented on HADOOP-6803:
-----------------------------------
+1 Looks good.
> Add native gzip read/write coverage to TestCodec
> -------------------------------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Test
> Components: io
> Affects Versions: 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.22.0
>
> Attachments: hadoop-6803-1.patch, hadoop-6803-2.patch, hadoop-6803-3.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6803) Add native gzip read/write coverage
to TestCodec
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-6803:
--------------------------------
Status: Patch Available (was: Open)
> Add native gzip read/write coverage to TestCodec
> -------------------------------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Test
> Components: io
> Affects Versions: 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.22.0
>
> Attachments: hadoop-6803-1.patch, hadoop-6803-2.patch, hadoop-6803-3.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6803) ZlibCompressor hangs on close
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-6803:
--------------------------------
Attachment: hadoop-6803-1.patch
Patch attached.
> ZlibCompressor hangs on close
> -----------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.21.0, 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.21.0, 0.22.0
>
> Attachments: hadoop-6803-1.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6803) Add native gzip read/write coverage
to TestCodec
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12874880#action_12874880 ]
Eli Collins commented on HADOOP-6803:
-------------------------------------
I figured it out. In the earlier change I had remove the synchronized modifier from ZlibCompressor#finished which would cause the jvm to miss the update from the jni code.
> Add native gzip read/write coverage to TestCodec
> -------------------------------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Test
> Components: io
> Affects Versions: 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.22.0
>
> Attachments: hadoop-6803-1.patch, hadoop-6803-2.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6803) Add native gzip read/write coverage
to TestCodec
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HADOOP-6803:
------------------------------
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Resolution: Fixed
I've just committed this. Thanks Eli!
> Add native gzip read/write coverage to TestCodec
> -------------------------------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Test
> Components: io
> Affects Versions: 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.22.0
>
> Attachments: hadoop-6803-1.patch, hadoop-6803-2.patch, hadoop-6803-3.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6803) Add native gzip read/write coverage
to TestCodec
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-6803:
--------------------------------
Attachment: hadoop-6803-3.patch
Updated patch resolves with trunk.
> Add native gzip read/write coverage to TestCodec
> -------------------------------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Test
> Components: io
> Affects Versions: 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.22.0
>
> Attachments: hadoop-6803-1.patch, hadoop-6803-2.patch, hadoop-6803-3.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6803) Add native gzip read/write coverage
to TestCodec
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900068#action_12900068 ]
Hudson commented on HADOOP-6803:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #360 (See [https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/360/])
HADOOP-6803. Add native gzip read/write coverage to TestCodec. Contributed by Eli Collins.
> Add native gzip read/write coverage to TestCodec
> -------------------------------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Test
> Components: io
> Affects Versions: 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.22.0
>
> Attachments: hadoop-6803-1.patch, hadoop-6803-2.patch, hadoop-6803-3.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6803) ZlibCompressor hangs on close
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12874874#action_12874874 ]
Eli Collins commented on HADOOP-6803:
-------------------------------------
Your right, I forgot that methods not fields have the "native" modifier. The "finish" field is read from the native code and "finished" is updated there so it looks like it should work. What's strange is if I revert the change to ZlibCompressor it passes now, earlier when I first added the test and ran with the native lib it hung with jstack showing the following:
{code}
"main" prio=10 tid=0x0000000040b19000 nid=0x36f0 runnable [0x00007f1257588000]
java.lang.Thread.State: RUNNABLE
at org.apache.hadoop.io.compress.zlib.ZlibCompressor.deflateBytesDirect(Native Method)
at org.apache.hadoop.io.compress.zlib.ZlibCompressor.compress(ZlibCompressor.java:359)
- locked <0x00007f12495063e0> (a org.apache.hadoop.io.compress.GzipCodec$GzipZlibCompressor)
at org.apache.hadoop.io.compress.CompressorStream.compress(CompressorStream.java:76)
at org.apache.hadoop.io.compress.CompressorStream.finish(CompressorStream.java:86)
at org.apache.hadoop.io.compress.CompressorStream.close(CompressorStream.java:97)
at sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:301)
at sun.nio.cs.StreamEncoder.close(StreamEncoder.java:130)
- locked <0x00007f12495167c8> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.close(OutputStreamWriter.java:216)
at java.io.BufferedWriter.close(BufferedWriter.java:248)
- locked <0x00007f12495167c8> (a java.io.OutputStreamWriter)
at org.apache.hadoop.io.compress.TestCodec.testGzipCodecWrite(TestCodec.java:653)
{code}
> ZlibCompressor hangs on close
> -----------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.21.0, 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.21.0, 0.22.0
>
> Attachments: hadoop-6803-1.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6803) Add native gzip read/write coverage
to TestCodec
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900250#action_12900250 ]
Hudson commented on HADOOP-6803:
--------------------------------
Integrated in Hadoop-Common-trunk #428 (See [https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/428/])
HADOOP-6803. Add native gzip read/write coverage to TestCodec. Contributed by Eli Collins.
> Add native gzip read/write coverage to TestCodec
> -------------------------------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Test
> Components: io
> Affects Versions: 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.22.0
>
> Attachments: hadoop-6803-1.patch, hadoop-6803-2.patch, hadoop-6803-3.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6803) ZlibCompressor hangs on close
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12874861#action_12874861 ]
Chris Douglas commented on HADOOP-6803:
---------------------------------------
Isn't the {{finished}} member set by the JNI code?
> ZlibCompressor hangs on close
> -----------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.21.0, 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.21.0, 0.22.0
>
> Attachments: hadoop-6803-1.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6803) Add native gzip read/write coverage
to TestCodec
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12875261#action_12875261 ]
Hadoop QA commented on HADOOP-6803:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12446204/hadoop-6803-2.patch
against trunk revision 951081.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/75/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/75/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/75/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/75/console
This message is automatically generated.
> Add native gzip read/write coverage to TestCodec
> -------------------------------------------------
>
> Key: HADOOP-6803
> URL: https://issues.apache.org/jira/browse/HADOOP-6803
> Project: Hadoop Common
> Issue Type: Test
> Components: io
> Affects Versions: 0.22.0
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 0.22.0
>
> Attachments: hadoop-6803-1.patch, hadoop-6803-2.patch
>
>
> Looking at ZlibCompressor I noticed that the finished member is never modified, and is therefore always false. This means ZlibCompressor#finished will always return false so CompressorStream#close loops indefinitely in finish:
> {code}
> while (!compressor.finished()) {
> compress();
> }
> {code}
> I modifed TestCodec#testGzipCodecWrite to also cover writing using the native lib and confirmed the hang with jstack. The fix is simple, ZlibCompressor should record when it's been finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.