You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2020/05/09 03:06:00 UTC
[jira] [Updated] (HUDI-786) InlineFileSystem.read API should ensure
content beyond inline length gets an EOF
[ https://issues.apache.org/jira/browse/HUDI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
sivabalan narayanan updated HUDI-786:
-------------------------------------
Labels: bug-bash-0.6.0 pull-request-available (was: pull-request-available)
> InlineFileSystem.read API should ensure content beyond inline length gets an EOF
> --------------------------------------------------------------------------------
>
> Key: HUDI-786
> URL: https://issues.apache.org/jira/browse/HUDI-786
> Project: Apache Hudi (incubating)
> Issue Type: Bug
> Components: Common Core
> Reporter: Vinoth Chandar
> Assignee: sivabalan narayanan
> Priority: Major
> Labels: bug-bash-0.6.0, pull-request-available
> Fix For: 0.6.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> While trying to investigate a flaky test, noticed that the readFully() just proceeds to read bytes from the outerStream without any bounds checking
> {code}
> @Override
> public void readFully(long position, byte[] buffer, int offset, int length) throws IOException {
> if ((length - offset) > this.length) {
> throw new IOException("Attempting to read past inline content");
> }
> outerStream.readFully(startOffset + position, buffer, offset, length);
> }
> @Override
> public void readFully(long position, byte[] buffer)
> throws IOException {
> readFully(position, buffer, 0, buffer.length);
> }
> {code}
> we need to throw an error for buffers that are trying to read past the inline content.. (potentially buggy) example shown above.
> I have also ignored the TestInlineFileSystem#testFileSystemAPIs() ... we need to make a change to respect suffix length (we randomly generate) while attempting to read past the 1000 bytes of inline content..
> {code}
> actualBytes = new byte[1000 + outerPathInfo.suffixLength];
> fsDataInputStream.readFully(0, actualBytes);
> verifyArrayEquality(outerPathInfo.expectedBytes, 0, 1000, actualBytes, 0, 1000);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)