[jira] [Created] (HDFS-10512) VolumeScanner may terminate to due NPE in DataNode.reportBadBlocks

["Wei-Chiu Chuang (JIRA)" <ji...@apache.org>]

Wei-Chiu Chuang created HDFS-10512:
--------------------------------------

             Summary: VolumeScanner may terminate to due NPE in DataNode.reportBadBlocks
                 Key: HDFS-10512
                 URL: https://issues.apache.org/jira/browse/HDFS-10512
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: datanode
            Reporter: Wei-Chiu Chuang


VolumeScanner may terminate due to unexpected NullPointerException thrown in {{DataNode.reportBadBlocks()}}

I observed this bug in a production CDH 5.5.1 cluster and the same bug still persist in upstream trunk.

{noformat}
2016-04-07 20:30:53,830 WARN org.apache.hadoop.hdfs.server.datanode.VolumeScanner: Reporting bad BP-1800173197-10.204.68.5-1444425156296:blk_1170134484_96468685 on /dfs/dn
2016-04-07 20:30:53,831 ERROR org.apache.hadoop.hdfs.server.datanode.VolumeScanner: VolumeScanner(/dfs/dn, DS-89b72832-2a8c-48f3-8235-48e6c5eb5ab3) exiting because of exception
java.lang.NullPointerException
        at org.apache.hadoop.hdfs.server.datanode.DataNode.reportBadBlocks(DataNode.java:1018)
        at org.apache.hadoop.hdfs.server.datanode.VolumeScanner$ScanResultHandler.handle(VolumeScanner.java:287)
        at org.apache.hadoop.hdfs.server.datanode.VolumeScanner.scanBlock(VolumeScanner.java:443)
        at org.apache.hadoop.hdfs.server.datanode.VolumeScanner.runLoop(VolumeScanner.java:547)
        at org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621)
2016-04-07 20:30:53,832 INFO org.apache.hadoop.hdfs.server.datanode.VolumeScanner: VolumeScanner(/dfs/dn, DS-89b72832-2a8c-48f3-8235-48e6c5eb5ab3) exiting.
{noformat}

I think the NPE comes from the volume variable in the following code snippet. Somehow the volume scanner know the volume, but the datanode can not lookup the volume using the block.
{code}
public void reportBadBlocks(ExtendedBlock block) throws IOException{
    BPOfferService bpos = getBPOSForBlock(block);
    FsVolumeSpi volume = getFSDataset().getVolume(block);
    bpos.reportBadBlocks(
        block, volume.getStorageID(), volume.getStorageType());
  }
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org