[jira] [Comment Edited] (YARN-8435) NullPointerException when client first time connect to Yarn Router

["rangjiaheng (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16517395#comment-16517395 ] 

rangjiaheng edited comment on YARN-8435 at 6/19/18 6:20 PM:
------------------------------------------------------------

 
{code:java}
private RequestInterceptorChainWrapper getInterceptorChain()
    throws IOException {
  String user = UserGroupInformation.getCurrentUser().getUserName();
  if (!userPipelineMap.containsKey(user)) {
    initializePipeline(user);
  }
  return userPipelineMap.get(user);
}

private void initializePipeline(String user) {
  RequestInterceptorChainWrapper chainWrapper = null;
  synchronized (this.userPipelineMap) {
    if (this.userPipelineMap.containsKey(user)) {
      LOG.info("Request to start an already existing user: {}"
          + " was received, so ignoring.", user);
      return;
    }

    chainWrapper = new RequestInterceptorChainWrapper();
    this.userPipelineMap.put(user, chainWrapper);
  }

  // We register the pipeline instance in the map first and then initialize it
  // later because chain initialization can be expensive and we would like to
  // release the lock as soon as possible to prevent other applications from
  // blocking when one application's chain is initializing
  LOG.info("Initializing request processing pipeline for application "
      + "for the user: {}", user);

  try {
    ClientRequestInterceptor interceptorChain =
        this.createRequestInterceptorChain();
    interceptorChain.init(user);
    chainWrapper.init(interceptorChain);
  } catch (Exception e) {
    synchronized (this.userPipelineMap) {
      this.userPipelineMap.remove(user);
    }
    throw e;
  }
}
{code}
 

As we can see, when two client process begin to connect to router, they may call getInterceptorChain() at the same time. Suppose the first thread get nothing from userPipelineMap, it then call initializePipeline(), build a chainWrapper and put it in userPipelineMap in synchronized (this.userPipelineMap) {}, now the second thread may get chainWrapper out from userPipelineMap, but it is not init !

Then the second thread call pipeline.getRootInterceptor().submitApplication(), a NullPointerException happened since pipeline.getRootInterceptor() was null.

 


was (Author: neomatrix):
 
{code:java}
private RequestInterceptorChainWrapper getInterceptorChain()
    throws IOException {
  String user = UserGroupInformation.getCurrentUser().getUserName();
  if (!userPipelineMap.containsKey(user)) {
    initializePipeline(user);
  }
  return userPipelineMap.get(user);
}

private void initializePipeline(String user) {
  RequestInterceptorChainWrapper chainWrapper = null;
  synchronized (this.userPipelineMap) {
    if (this.userPipelineMap.containsKey(user)) {
      LOG.info("Request to start an already existing user: {}"
          + " was received, so ignoring.", user);
      return;
    }

    chainWrapper = new RequestInterceptorChainWrapper();
    this.userPipelineMap.put(user, chainWrapper);
  }

  // We register the pipeline instance in the map first and then initialize it
  // later because chain initialization can be expensive and we would like to
  // release the lock as soon as possible to prevent other applications from
  // blocking when one application's chain is initializing
  LOG.info("Initializing request processing pipeline for application "
      + "for the user: {}", user);

  try {
    ClientRequestInterceptor interceptorChain =
        this.createRequestInterceptorChain();
    interceptorChain.init(user);
    chainWrapper.init(interceptorChain);
  } catch (Exception e) {
    synchronized (this.userPipelineMap) {
      this.userPipelineMap.remove(user);
    }
    throw e;
  }
}
{code}
 

As we can see, when two client process begin to connect to router, they may call getInterceptorChain() at the same time. Suppose the first thread get nothing from userPipelineMap, it then call initializePipeline(), build a chainWrapper and put it in userPipelineMap in synchronized (this.userPipelineMap) {}, now the second thread may get chainWrapper out from userPipelineMap, but it is not init !

Then the second thread call pipeline.getRootInterceptor().submitApplication(), a NullPointerException happened since pipeline.getRootInterceptor() was null.

 

 

> NullPointerException when client first time connect to Yarn Router
> ------------------------------------------------------------------
>
>                 Key: YARN-8435
>                 URL: https://issues.apache.org/jira/browse/YARN-8435
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: router
>    Affects Versions: 3.0.2
>            Reporter: rangjiaheng
>            Priority: Critical
>
> When Two client process (with the same user name and the same hostname) begin to connect to yarn router at the same time, to submit application, kill application, ... and so on, then a java.lang.NullPointerException may throws from yarn router.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org