[jira] [Commented] (TEZ-2289) ATSHistoryLoggingService can generate ArrayOutOfBoundsException

["Chang Li (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-2289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14485311#comment-14485311 ] 

Chang Li commented on TEZ-2289:
-------------------------------

the exception is caused by the following code
{code}
int count = response.getErrors().size();
        for (int i = 0; i < count; ++i) {
          TimelinePutError err = response.getErrors().get(i);
          if (err.getErrorCode() != 0) {
            LOG.warn("Could not post history event to ATS"
                + ", atsPutError=" + err.getErrorCode()
                + ", entityId=" + entities[i].getEntityId()
                + ", eventType=" + events.get(i).getHistoryEvent().getEventType());
          }
        }
{code}
The current code assumes that the error size is equal to entities size and events size, which is obviously false, because not every entity and events would trigger error. Also due to related entities, the total errors could be larger than the size of entities. In the first scenario the entity and eventType retireved is incorrect, in the second scenario the ArrayIndexOutOfBoundsException happens. entityId could be correctly retrieved from err. eventType should be dropped in the warn message to avoid retrieving incorrectly or outofboundary issue.

> ATSHistoryLoggingService can generate ArrayOutOfBoundsException
> ---------------------------------------------------------------
>
>                 Key: TEZ-2289
>                 URL: https://issues.apache.org/jira/browse/TEZ-2289
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Jonathan Eagles
>            Assignee: Chang Li
>         Attachments: TEZ-2289.1.patch, TEZ-2289.patch
>
>
> 2015-04-07 23:11:20,459 INFO [main] app.DAGAppMaster: Running DAG: MRRSleepJob
> 2015-04-07 23:11:20,546 INFO [IPC Server handler 0 on 50500] ipc.Server: IPC Server handler 0 on 50500, call org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus from 127.0.0.1:53151 Call#93 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
>         at org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
>         at org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
>         at org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
>         at org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
>         at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1694)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)
> 2015-04-07 23:11:20,875 INFO [main] history.HistoryEventHandler: [HISTORY][DAG:dag_1427297554817_0149_1][Event:DAG_SUBMITTED]: dagID=dag_1427297554817_0149_1, submitTime=1428448280397
> 2015-04-07 23:11:20,905 WARN [HistoryEventHandlingThread] ats.ATSHistoryLoggingService: Could not post history event to ATS, atsPutError=6, entityId=dag_1427297554817_0149_1, eventType=DAG_SUBMITTED
> 2015-04-07 23:11:20,906 WARN [HistoryEventHandlingThread] ats.ATSHistoryLoggingService: Could not handle history events
> java.lang.ArrayIndexOutOfBoundsException: 1
>         at org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.handleEvents(ATSHistoryLoggingService.java:312)
>         at org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.access$700(ATSHistoryLoggingService.java:50)
>         at org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService$1.run(ATSHistoryLoggingService.java:159)
>         at java.lang.Thread.run(Thread.java:722)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)