[jira] [Commented] (YARN-4818) AggregatedLogFormat.LogValue.write() incorrectly truncates files

["Jason Lowe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196297#comment-15196297 ] 

Jason Lowe commented on YARN-4818:
----------------------------------

Is there a recent change in 2.8 that broke this?  I haven't seen it in 2.7, yet it too is using the buffer to copy the bytes in a similar manner and I haven't heard of any reports from our users about truncated logs.  The FileInputStream claim in the description above isn't mentioned in the javadocs, and was I not able to replicate it with a simple cat-like program:
{code}
import java.io.File;
import java.io.FileInputStream;
class cat {
    public static void main(String[] args) throws Exception {
	byte[] buf = new byte[65536];
	File f = new File(args[0]);
	FileInputStream in = new FileInputStream(f);
	int len;
	while ((len = in.read(buf)) != -1) {
	    System.out.write(buf, 0, len);
	}
    }
}
{code}

I'm able to use the code above to successfully copy files of arbitrary sizes with JDK7 on Linux.  Could this be a platform-specific issue?

> AggregatedLogFormat.LogValue.write() incorrectly truncates files
> ----------------------------------------------------------------
>
>                 Key: YARN-4818
>                 URL: https://issues.apache.org/jira/browse/YARN-4818
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.8.0
>            Reporter: Brook Zhou
>            Assignee: Brook Zhou
>              Labels: log-aggregation
>             Fix For: 2.8.0
>
>         Attachments: YARN-4818-v0.patch
>
>
> AggregatedLogFormat.LogValue.write() currently has a bug where it only writes in blocks of the buffer size (65535). This is because FileInputStream.read(byte[] buf) returns -1 if there are less than buf.length bytes remaining. In cases where the file size is not an exact multiple of 65535 bytes, the remaining bytes are truncated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)