You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Doug Cutting (JIRA)" <ji...@apache.org> on 2010/02/06 00:18:28 UTC
[jira] Created: (AVRO-407) java data interop tests fail
java data interop tests fail
----------------------------
Key: AVRO-407
URL: https://issues.apache.org/jira/browse/AVRO-407
Project: Avro
Issue Type: Bug
Components: java
Reporter: Doug Cutting
Priority: Blocker
Fix For: 1.3.0
Java data file interop tests currently fail.
If you run: cd lang/java; ant interop-data-generate interop-data-test
Then the test log shows:
java.io.IOException: Incomplete Block
org.apache.avro.AvroRuntimeException: java.io.IOException: Incomplete Block
at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:169)
at org.apache.avro.TestDataFile.readFile(TestDataFile.java:193)
at org.apache.avro.TestDataFile$InteropTest.readFiles(TestDataFile.java:238)
at org.apache.avro.TestDataFile$InteropTest.testGeneratedGeneric(TestDataFile.java:216)
Caused by: java.io.IOException: Incomplete Block
at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:161)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-407) java data interop tests fail
Posted by "Scott Carey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Scott Carey updated AVRO-407:
-----------------------------
Attachment: AVRO-407.patch
This patch fixes the issue.
> java data interop tests fail
> ----------------------------
>
> Key: AVRO-407
> URL: https://issues.apache.org/jira/browse/AVRO-407
> Project: Avro
> Issue Type: Bug
> Components: java
> Reporter: Doug Cutting
> Priority: Blocker
> Fix For: 1.3.0
>
> Attachments: AVRO-407.patch
>
>
> Java data file interop tests currently fail.
> If you run: cd lang/java; ant interop-data-generate interop-data-test
> Then the test log shows:
> java.io.IOException: Incomplete Block
> org.apache.avro.AvroRuntimeException: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:169)
> at org.apache.avro.TestDataFile.readFile(TestDataFile.java:193)
> at org.apache.avro.TestDataFile$InteropTest.readFiles(TestDataFile.java:238)
> at org.apache.avro.TestDataFile$InteropTest.testGeneratedGeneric(TestDataFile.java:216)
> Caused by: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:161)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-407) java data interop tests fail
Posted by "Scott Carey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830392#action_12830392 ]
Scott Carey commented on AVRO-407:
----------------------------------
I don't know what it is.
I refactored DataFileStream to have a nextBlock() and hasNextBlock() method and during that resolved some unit test failures with similar stack traces to the above. I'll check out from trunk separately and see if it looks familiar.
> java data interop tests fail
> ----------------------------
>
> Key: AVRO-407
> URL: https://issues.apache.org/jira/browse/AVRO-407
> Project: Avro
> Issue Type: Bug
> Components: java
> Reporter: Doug Cutting
> Priority: Blocker
> Fix For: 1.3.0
>
>
> Java data file interop tests currently fail.
> If you run: cd lang/java; ant interop-data-generate interop-data-test
> Then the test log shows:
> java.io.IOException: Incomplete Block
> org.apache.avro.AvroRuntimeException: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:169)
> at org.apache.avro.TestDataFile.readFile(TestDataFile.java:193)
> at org.apache.avro.TestDataFile$InteropTest.readFiles(TestDataFile.java:238)
> at org.apache.avro.TestDataFile$InteropTest.testGeneratedGeneric(TestDataFile.java:216)
> Caused by: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:161)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-407) java data interop tests fail
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830442#action_12830442 ]
Philip Zeyliger commented on AVRO-407:
--------------------------------------
Patch looks correct. (I was a bit worried about this in https://issues.apache.org/jira/browse/AVRO-380?focusedCommentId=12806025&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12806025, but failed to write a unit test to convince myself.)
Please do add a unit test that triggers this.
-- Philip
> java data interop tests fail
> ----------------------------
>
> Key: AVRO-407
> URL: https://issues.apache.org/jira/browse/AVRO-407
> Project: Avro
> Issue Type: Bug
> Components: java
> Reporter: Doug Cutting
> Priority: Blocker
> Fix For: 1.3.0
>
> Attachments: AVRO-407.patch
>
>
> Java data file interop tests currently fail.
> If you run: cd lang/java; ant interop-data-generate interop-data-test
> Then the test log shows:
> java.io.IOException: Incomplete Block
> org.apache.avro.AvroRuntimeException: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:169)
> at org.apache.avro.TestDataFile.readFile(TestDataFile.java:193)
> at org.apache.avro.TestDataFile$InteropTest.readFiles(TestDataFile.java:238)
> at org.apache.avro.TestDataFile$InteropTest.testGeneratedGeneric(TestDataFile.java:216)
> Caused by: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:161)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-407) java data interop tests fail
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-407:
------------------------------
Resolution: Fixed
Assignee: Scott Carey
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Scott!
> java data interop tests fail
> ----------------------------
>
> Key: AVRO-407
> URL: https://issues.apache.org/jira/browse/AVRO-407
> Project: Avro
> Issue Type: Bug
> Components: java
> Reporter: Doug Cutting
> Assignee: Scott Carey
> Priority: Blocker
> Fix For: 1.3.0
>
> Attachments: AVRO-407.patch
>
>
> Java data file interop tests currently fail.
> If you run: cd lang/java; ant interop-data-generate interop-data-test
> Then the test log shows:
> java.io.IOException: Incomplete Block
> org.apache.avro.AvroRuntimeException: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:169)
> at org.apache.avro.TestDataFile.readFile(TestDataFile.java:193)
> at org.apache.avro.TestDataFile$InteropTest.readFiles(TestDataFile.java:238)
> at org.apache.avro.TestDataFile$InteropTest.testGeneratedGeneric(TestDataFile.java:216)
> Caused by: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:161)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-407) java data interop tests fail
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830372#action_12830372 ]
Doug Cutting commented on AVRO-407:
-----------------------------------
Do you know what the problem is? It would be good to fix this independently ASAP, since it makes it harder for C and python folks to check interoperability when Java's interop tests fail.
> java data interop tests fail
> ----------------------------
>
> Key: AVRO-407
> URL: https://issues.apache.org/jira/browse/AVRO-407
> Project: Avro
> Issue Type: Bug
> Components: java
> Reporter: Doug Cutting
> Priority: Blocker
> Fix For: 1.3.0
>
>
> Java data file interop tests currently fail.
> If you run: cd lang/java; ant interop-data-generate interop-data-test
> Then the test log shows:
> java.io.IOException: Incomplete Block
> org.apache.avro.AvroRuntimeException: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:169)
> at org.apache.avro.TestDataFile.readFile(TestDataFile.java:193)
> at org.apache.avro.TestDataFile$InteropTest.readFiles(TestDataFile.java:238)
> at org.apache.avro.TestDataFile$InteropTest.testGeneratedGeneric(TestDataFile.java:216)
> Caused by: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:161)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-407) java data interop tests fail
Posted by "Scott Carey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Scott Carey updated AVRO-407:
-----------------------------
Status: Patch Available (was: Open)
> java data interop tests fail
> ----------------------------
>
> Key: AVRO-407
> URL: https://issues.apache.org/jira/browse/AVRO-407
> Project: Avro
> Issue Type: Bug
> Components: java
> Reporter: Doug Cutting
> Priority: Blocker
> Fix For: 1.3.0
>
> Attachments: AVRO-407.patch
>
>
> Java data file interop tests currently fail.
> If you run: cd lang/java; ant interop-data-generate interop-data-test
> Then the test log shows:
> java.io.IOException: Incomplete Block
> org.apache.avro.AvroRuntimeException: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:169)
> at org.apache.avro.TestDataFile.readFile(TestDataFile.java:193)
> at org.apache.avro.TestDataFile$InteropTest.readFiles(TestDataFile.java:238)
> at org.apache.avro.TestDataFile$InteropTest.testGeneratedGeneric(TestDataFile.java:216)
> Caused by: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:161)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-407) java data interop tests fail
Posted by "Scott Carey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830369#action_12830369 ]
Scott Carey commented on AVRO-407:
----------------------------------
FYI: I am working on some enhancements to the Java DataFileWriter and DataFileReader, and on my working copy these tests succeed.
I plan on submitting the enhancements soon, which could clear this up as well.
> java data interop tests fail
> ----------------------------
>
> Key: AVRO-407
> URL: https://issues.apache.org/jira/browse/AVRO-407
> Project: Avro
> Issue Type: Bug
> Components: java
> Reporter: Doug Cutting
> Priority: Blocker
> Fix For: 1.3.0
>
>
> Java data file interop tests currently fail.
> If you run: cd lang/java; ant interop-data-generate interop-data-test
> Then the test log shows:
> java.io.IOException: Incomplete Block
> org.apache.avro.AvroRuntimeException: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:169)
> at org.apache.avro.TestDataFile.readFile(TestDataFile.java:193)
> at org.apache.avro.TestDataFile$InteropTest.readFiles(TestDataFile.java:238)
> at org.apache.avro.TestDataFile$InteropTest.testGeneratedGeneric(TestDataFile.java:216)
> Caused by: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:161)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-407) java data interop tests fail
Posted by "Scott Carey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830414#action_12830414 ]
Scott Carey commented on AVRO-407:
----------------------------------
The file was written fine, the reading of the file did not account for inputStream returning with a partial read. We probably need to add a unit test that writes and reads a file with larger blocks, currently the tests all use small block sizes.
> java data interop tests fail
> ----------------------------
>
> Key: AVRO-407
> URL: https://issues.apache.org/jira/browse/AVRO-407
> Project: Avro
> Issue Type: Bug
> Components: java
> Reporter: Doug Cutting
> Priority: Blocker
> Fix For: 1.3.0
>
> Attachments: AVRO-407.patch
>
>
> Java data file interop tests currently fail.
> If you run: cd lang/java; ant interop-data-generate interop-data-test
> Then the test log shows:
> java.io.IOException: Incomplete Block
> org.apache.avro.AvroRuntimeException: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:169)
> at org.apache.avro.TestDataFile.readFile(TestDataFile.java:193)
> at org.apache.avro.TestDataFile$InteropTest.readFiles(TestDataFile.java:238)
> at org.apache.avro.TestDataFile$InteropTest.testGeneratedGeneric(TestDataFile.java:216)
> Caused by: java.io.IOException: Incomplete Block
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:161)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.