You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Raghu Angadi (JIRA)" <ji...@apache.org> on 2007/01/18 19:31:29 UTC

[jira] Created: (HADOOP-902) NPE in DFSOutputStream.closeBackupStream()

NPE in DFSOutputStream.closeBackupStream()
------------------------------------------

                 Key: HADOOP-902
                 URL: https://issues.apache.org/jira/browse/HADOOP-902
             Project: Hadoop
          Issue Type: Bug
          Components: dfs
            Reporter: Raghu Angadi
         Assigned To: Raghu Angadi
             Fix For: 0.11.0


After HADOOP-757 is checked in, Devraj noticed following NPE:

java.lang.NullPointerException
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.closeBackupStream(DFSClient.java:972)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1219)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1181)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1163)
at org.apache.hadoop.fs.FSDataOutputStream$Summer.write(FSDataOutputStream.java:85)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:114)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at java.io.FilterOutputStream.close(FilterOutputStream.java:140)
at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:558)
at org.apache.hadoop.mapred.SequenceFileOutputFormat$1.close(SequenceFileOutputFormat.java:72)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:331)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1367)

I will submit a patch after confirming its ok for backupStream to be null at this stage.


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] Updated: (HADOOP-902) NPE in DFSOutputStream.closeBackupStream()

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-902:
--------------------------------

    Attachment: HADOOP-902.patch

> NPE in DFSOutputStream.closeBackupStream()
> ------------------------------------------
>
>                 Key: HADOOP-902
>                 URL: https://issues.apache.org/jira/browse/HADOOP-902
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: Raghu Angadi
>         Assigned To: Raghu Angadi
>             Fix For: 0.11.0
>
>         Attachments: HADOOP-902.patch
>
>
> After HADOOP-757 is checked in, Devraj noticed following NPE:
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.closeBackupStream(DFSClient.java:972)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1219)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1181)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1163)
> at org.apache.hadoop.fs.FSDataOutputStream$Summer.write(FSDataOutputStream.java:85)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:114)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
> at java.io.FilterOutputStream.close(FilterOutputStream.java:140)
> at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:558)
> at org.apache.hadoop.mapred.SequenceFileOutputFormat$1.close(SequenceFileOutputFormat.java:72)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:331)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1367)
> I will submit a patch after confirming its ok for backupStream to be null at this stage.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] Commented: (HADOOP-902) NPE in DFSOutputStream.closeBackupStream()

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465899 ] 

Hadoop QA commented on HADOOP-902:
----------------------------------

+1, because http://issues.apache.org/jira/secure/attachment/12349210/HADOOP-902.patch applied and successfully tested against trunk revision r497583.

> NPE in DFSOutputStream.closeBackupStream()
> ------------------------------------------
>
>                 Key: HADOOP-902
>                 URL: https://issues.apache.org/jira/browse/HADOOP-902
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: Raghu Angadi
>         Assigned To: Raghu Angadi
>             Fix For: 0.11.0
>
>         Attachments: HADOOP-902.patch
>
>
> After HADOOP-757 is checked in, Devraj noticed following NPE:
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.closeBackupStream(DFSClient.java:972)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1219)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1181)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1163)
> at org.apache.hadoop.fs.FSDataOutputStream$Summer.write(FSDataOutputStream.java:85)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:114)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
> at java.io.FilterOutputStream.close(FilterOutputStream.java:140)
> at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:558)
> at org.apache.hadoop.mapred.SequenceFileOutputFormat$1.close(SequenceFileOutputFormat.java:72)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:331)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1367)
> I will submit a patch after confirming its ok for backupStream to be null at this stage.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] Updated: (HADOOP-902) NPE in DFSOutputStream.closeBackupStream()

Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doug Cutting updated HADOOP-902:
--------------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

I just committed this.  Thanks, Raghu!

> NPE in DFSOutputStream.closeBackupStream()
> ------------------------------------------
>
>                 Key: HADOOP-902
>                 URL: https://issues.apache.org/jira/browse/HADOOP-902
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: Raghu Angadi
>         Assigned To: Raghu Angadi
>             Fix For: 0.11.0
>
>         Attachments: HADOOP-902.patch
>
>
> After HADOOP-757 is checked in, Devraj noticed following NPE:
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.closeBackupStream(DFSClient.java:972)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1219)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1181)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1163)
> at org.apache.hadoop.fs.FSDataOutputStream$Summer.write(FSDataOutputStream.java:85)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:114)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
> at java.io.FilterOutputStream.close(FilterOutputStream.java:140)
> at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:558)
> at org.apache.hadoop.mapred.SequenceFileOutputFormat$1.close(SequenceFileOutputFormat.java:72)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:331)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1367)
> I will submit a patch after confirming its ok for backupStream to be null at this stage.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] Commented: (HADOOP-902) NPE in DFSOutputStream.closeBackupStream()

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465844 ] 

Raghu Angadi commented on HADOOP-902:
-------------------------------------

backupStream could be null when close() is called multiple times.. if the current write position is at the end of a block size, endBlock() is called before flushing any data which results in this NPE.

will submit a patch soon.





> NPE in DFSOutputStream.closeBackupStream()
> ------------------------------------------
>
>                 Key: HADOOP-902
>                 URL: https://issues.apache.org/jira/browse/HADOOP-902
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: Raghu Angadi
>         Assigned To: Raghu Angadi
>             Fix For: 0.11.0
>
>
> After HADOOP-757 is checked in, Devraj noticed following NPE:
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.closeBackupStream(DFSClient.java:972)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1219)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1181)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1163)
> at org.apache.hadoop.fs.FSDataOutputStream$Summer.write(FSDataOutputStream.java:85)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:114)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
> at java.io.FilterOutputStream.close(FilterOutputStream.java:140)
> at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:558)
> at org.apache.hadoop.mapred.SequenceFileOutputFormat$1.close(SequenceFileOutputFormat.java:72)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:331)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1367)
> I will submit a patch after confirming its ok for backupStream to be null at this stage.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] Updated: (HADOOP-902) NPE in DFSOutputStream.closeBackupStream()

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-902:
--------------------------------

    Status: Patch Available  (was: Open)


attached patch checks if backupStream or backupFile is null when each is closed.


> NPE in DFSOutputStream.closeBackupStream()
> ------------------------------------------
>
>                 Key: HADOOP-902
>                 URL: https://issues.apache.org/jira/browse/HADOOP-902
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: Raghu Angadi
>         Assigned To: Raghu Angadi
>             Fix For: 0.11.0
>
>         Attachments: HADOOP-902.patch
>
>
> After HADOOP-757 is checked in, Devraj noticed following NPE:
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.closeBackupStream(DFSClient.java:972)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1219)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1181)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1163)
> at org.apache.hadoop.fs.FSDataOutputStream$Summer.write(FSDataOutputStream.java:85)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:114)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
> at java.io.FilterOutputStream.close(FilterOutputStream.java:140)
> at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:558)
> at org.apache.hadoop.mapred.SequenceFileOutputFormat$1.close(SequenceFileOutputFormat.java:72)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:331)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1367)
> I will submit a patch after confirming its ok for backupStream to be null at this stage.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira