[jira] [Commented] (TEZ-3242) Reduce bytearray copy with TezEvent Serialization and deserialization

["TezQA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273004#comment-15273004 ] 

TezQA commented on TEZ-3242:
----------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment
  http://issues.apache.org/jira/secure/attachment/12802495/TEZ-3242-1.patch
  against master revision c3b8b85.

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 1 new or modified test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.

    {color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version 3.0.1) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.

    {color:red}-1 core tests{color}.  The patch failed these unit tests in :
                   org.apache.tez.test.TestFaultTolerance

Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1698//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1698//console

This message is automatically generated.

> Reduce bytearray copy with TezEvent Serialization and deserialization
> ---------------------------------------------------------------------
>
>                 Key: TEZ-3242
>                 URL: https://issues.apache.org/jira/browse/TEZ-3242
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Rohini Palaniswamy
>            Assignee: Rohini Palaniswamy
>             Fix For: 0.7.2, 0.8.4
>
>         Attachments: TEZ-3242-1.patch
>
>
> Byte arrays are created for serializing protobuf messages and parsing them which creates lot of garbage when we have lot of events. 
> {code}
> java.lang.OutOfMemoryError: Java heap space
> 	at java.util.Arrays.copyOf(Arrays.java:3236)
> 	at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
> 	at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
> 	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
> 	at java.io.DataOutputStream.write(DataOutputStream.java:107)
> 	at java.io.FilterOutputStream.write(FilterOutputStream.java:97)
> 	at org.apache.tez.runtime.api.impl.TezEvent.serializeEvent(TezEvent.java:197)
> 	at org.apache.tez.runtime.api.impl.TezEvent.write(TezEvent.java:268)
> 	at org.apache.tez.runtime.api.impl.TezHeartbeatResponse.write(TezHeartbeatResponse.java:95)
> 	at org.apache.hadoop.io.ObjectWritable.writeObject(ObjectWritable.java:202)
> 	at org.apache.hadoop.io.ObjectWritable.writeObject(ObjectWritable.java:128)
> 	at org.apache.hadoop.io.ObjectWritable.write(ObjectWritable.java:82)
> 	at org.apache.hadoop.ipc.Server.setupResponse(Server.java:2496)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)