[jira] [Created] (BEAM-9743) TFRecordCodec not attempt to fully read header/footer

["Kyoungha Min (Jira)" <ji...@apache.org>]

Kyoungha Min created BEAM-9743:
----------------------------------

             Summary: TFRecordCodec not attempt to fully read header/footer
                 Key: BEAM-9743
                 URL: https://issues.apache.org/jira/browse/BEAM-9743
             Project: Beam
          Issue Type: Bug
          Components: sdk-java-core
            Reporter: Kyoungha Min
            Assignee: Kyoungha Min


Seems like it only happens with Zstd compression (or any other picky input stream that refuse to read fully). Zstd seems very picky at giving out data.

The parts with the issue are

[https://github.com/apache/beam/blob/c7911043510a266078a3dc8faef7a1dbe1f598c5/sdks/java/core/src/main/java/org/apache/beam/sdk/io/TFRecordIO.java#L672]

[https://github.com/apache/beam/blob/c7911043510a266078a3dc8faef7a1dbe1f598c5/sdks/java/core/src/main/java/org/apache/beam/sdk/io/TFRecordIO.java#L699]

 

And not so problem within the beam application, but still not following the WritableByteChannel API, 

[https://github.com/apache/beam/blob/c7911043510a266078a3dc8faef7a1dbe1f598c5/sdks/java/core/src/main/java/org/apache/beam/sdk/io/TFRecordIO.java#L720-L727]

 

ReadableByteChannel/WritableByteChannel Javadoc specifies that they are not required to read/write fully, and can refuse to read/write time to time.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)