You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "stack (JIRA)" <ji...@apache.org> on 2008/01/31 20:05:08 UTC
[jira] Created: (HADOOP-2756) [hbase] NPE in DFSClient in hbase
under load
[hbase] NPE in DFSClient in hbase under load
--------------------------------------------
Key: HADOOP-2756
URL: https://issues.apache.org/jira/browse/HADOOP-2756
Project: Hadoop Core
Issue Type: Bug
Components: contrib/hbase
Reporter: stack
Priority: Minor
Saw this in logs:
{code}
2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
java.lang.NullPointerException
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
{code}
Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-2756) NPE in DFSClient in hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi reassigned HADOOP-2756:
------------------------------------
Assignee: Raghu Angadi (was: dhruba borthakur)
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2756) NPE in DFSClient in hbase under
load
Posted by "Christian Kunz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12574799#action_12574799 ]
Christian Kunz commented on HADOOP-2756:
----------------------------------------
+1 blocker for 0.16.
Some of our tasks fail with this exception, not necessarily under load. And when trying to run distcp on the target cluster with about 20,000 mappers there were more than 400 such failures.
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Priority: Minor
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2756) NPE in DFSClient in hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-2756:
---------------------------------
Fix Version/s: 0.16.1
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-2756) NPE in DFSClient in
hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12568819#action_12568819 ]
rangadi edited comment on HADOOP-2756 at 2/13/08 11:03 PM:
----------------------------------------------------------------
we should probably be able to know that looking at the code. Any way, please feel free to change the 'fix version'. Personally I don't think we should have NPEs.
was (Author: rangadi):
please feel free to change the 'fix version'. Personally I don't think we should have NPEs.
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2756) NPE in DFSClient in hbase under
load
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12568783#action_12568783 ]
stack commented on HADOOP-2756:
-------------------------------
Will this fix the issue? It just moves the join closer to the streamer.close. If streamer can be null when we go join at line #2276 (before this patch is applied), why can't it be null just after the call to streamer.close?
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2756) NPE in DFSClient in hbase under
load
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12575192#action_12575192 ]
dhruba borthakur commented on HADOOP-2756:
------------------------------------------
I think we should go with this minimal patch for 0.16. +1.
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2756) NPE in DFSClient in hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-2756:
---------------------------------
Fix Version/s: (was: 0.16.1)
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Priority: Minor
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2756) NPE in DFSClient in hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-2756:
---------------------------------
Fix Version/s: 0.16.0
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: stack
> Priority: Minor
> Fix For: 0.16.0
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2756) NPE in DFSClient in hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-2756:
---------------------------------
Fix Version/s: 0.16.1
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2756) NPE in DFSClient in hbase under
load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12575219#action_12575219 ]
Raghu Angadi commented on HADOOP-2756:
--------------------------------------
If we prefer minimalist patch, then the first patch is fine too.
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch, HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2756) NPE in DFSClient in hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-2756:
---------------------------------
Component/s: (was: contrib/hbase)
dfs
Summary: NPE in DFSClient in hbase under load (was: [hbase] NPE in DFSClient in hbase under load)
This is probably a blocker for 0.16. Changed component to dfs. The fix might be similar to the fix in HADOOP-2717.
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: stack
> Priority: Minor
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2756) NPE in DFSClient in hbase under
load
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12575450#action_12575450 ]
dhruba borthakur commented on HADOOP-2756:
------------------------------------------
+1.
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch, HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2756) NPE in DFSClient in hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-2756:
---------------------------------
Attachment: HADOOP-2756.patch
Dhruba,
Could you review the updated patch?
This does not include a unit test since it is correlated more with sun spots :).
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch, HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-2756) NPE in DFSClient in hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi reassigned HADOOP-2756:
------------------------------------
Assignee: (was: Raghu Angadi)
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2756) NPE in DFSClient in hbase under
load
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12574824#action_12574824 ]
Jim Kellerman commented on HADOOP-2756:
---------------------------------------
+1 on this being a blocker for 0.16.1
We have seen this fairly often, but it is not reproducable. Seems to be related to sunspots or something.
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2756) NPE in DFSClient in hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-2756:
---------------------------------
Fix Version/s: (was: 0.16.0)
Assignee: dhruba borthakur
Affects Version/s: 0.16.0
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: dhruba borthakur
> Priority: Minor
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2756) NPE in DFSClient in hbase under
load
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12568808#action_12568808 ]
dhruba borthakur commented on HADOOP-2756:
------------------------------------------
The code looks correct but there could be an issue that needs more testing. The streamer thread might not exit (hence the streamer.join() might block indefinitely) if the reponseder thread is not closed (via response.close()).
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2756) NPE in DFSClient in hbase under
load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12568819#action_12568819 ]
Raghu Angadi commented on HADOOP-2756:
--------------------------------------
please feel free to change the 'fix version'. Personally I don't think we should have NPEs.
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2756) NPE in DFSClient in hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-2756:
---------------------------------
Attachment: HADOOP-2756.patch
Suggested patch.
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2756) NPE in DFSClient in hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-2756:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this.
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch, HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2756) NPE in DFSClient in hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-2756:
---------------------------------
Status: Patch Available (was: Open)
Thanks Dhruba. This patch applies to both 0.16 and trunk.
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch, HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2756) NPE in DFSClient in hbase under
load
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12575670#action_12575670 ]
Hudson commented on HADOOP-2756:
--------------------------------
Integrated in Hadoop-trunk #421 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/421/])
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch, HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2756) NPE in DFSClient in hbase under
load
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12575483#action_12575483 ]
Hadoop QA commented on HADOOP-2756:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12377133/HADOOP-2756.patch
against trunk revision 619744.
@author +1. The patch does not contain any @author tags.
tests included -1. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1896/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1896/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1896/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1896/console
This message is automatically generated.
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch, HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-2756) NPE in DFSClient in hbase under load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi reassigned HADOOP-2756:
------------------------------------
Assignee: Raghu Angadi
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2756) NPE in DFSClient in hbase under
load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12575136#action_12575136 ]
Raghu Angadi commented on HADOOP-2756:
--------------------------------------
> The streamer thread might not exit (hence the streamer.join() might block indefinitely) if the reponseder thread is not closed (via response.close()).
I don't see dataStreamer waiting for repsonse to close(). If it is not closed, it closes it.
I think attached patch is fine. I checked the peripheral code again and I don't see it introducing any new problem and fixes this bug. Does anyone see why it is wrong? If this fix is ok, I will submit a slightly reformatted patch (i.e. bring all the references to 'response' inside closeInternal() to one place).
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2756) NPE in DFSClient in hbase under
load
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12568789#action_12568789 ]
stack commented on HADOOP-2756:
-------------------------------
Or rather, why can't it be null after the test for null?
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2756) NPE in DFSClient in hbase under
load
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12568792#action_12568792 ]
Raghu Angadi commented on HADOOP-2756:
--------------------------------------
bq. If streamer can be null when we go join at line #2276 (before this patch is applied), why can't it be null just after the call to streamer.close?
NPE is with "response" object. I think streamer is the one that sets it to null asynchronously. I am just extending the fix in HADOOP-2717. This is as correct as the patch there.
> NPE in DFSClient in hbase under load
> ------------------------------------
>
> Key: HADOOP-2756
> URL: https://issues.apache.org/jira/browse/HADOOP-2756
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.16.1
>
> Attachments: HADOOP-2756.patch
>
>
> Saw this in logs:
> {code}
> 2008-01-31 18:55:02,128 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region TestTable,0009438931,1201805282651
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
> at org.apache.hadoop.hbase.HStoreFile.writeInfo(HStoreFile.java:365)
> at org.apache.hadoop.hbase.HStore.compact(HStore.java:1236)
> at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:775)
> at org.apache.hadoop.hbase.HRegion.compactIfNeeded(HRegion.java:707)
> at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:253)
> {code}
> Look to see if the response data method needs to be made volatile (There's a test for null just before we use it on line #2262).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.