You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Gaojinchao <ga...@huawei.com> on 2011/05/13 03:01:53 UTC
Re: [jira] [Updated] (HBASE-3878) Hbase client throws
NoSuchElementException
Hi Ted, Thanks for your patch.
I am not familiar with SoftValueSortedMap.
Please check this function.
void deleteCachedLocation(final byte [] tableName, final byte [] row) {
synchronized (this.cachedRegionLocations) {
SoftValueSortedMap<byte [], HRegionLocation> tableLocations =
getTableLocations(tableName);
// start to examine the cache. we can only do cache actions
// if there's something in the cache for this table.
if (!tableLocations.isEmpty()) {
HRegionLocation rl = getCachedLocation(tableName, row);
if (rl != null) {
tableLocations.remove(rl.getRegionInfo().getStartKey());
if (LOG.isDebugEnabled()) {
LOG.debug("Removed " +
rl.getRegionInfo().getRegionNameAsString() +
" for tableName=" + Bytes.toString(tableName) +
" from cache " + "because of " + Bytes.toStringBinary(row));
}
}
}
}
}
-----邮件原件-----
发件人: Ted Yu (JIRA) [mailto:jira@apache.org]
发送时间: 2011年5月12日 22:44
收件人: Gaojinchao
主题: [jira] [Updated] (HBASE-3878) Hbase client throws NoSuchElementException
[ https://issues.apache.org/jira/browse/HBASE-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-3878:
--------------------------
Attachment: 3878.patch
Added try/catch block to guard against NoSuchElementException.
We already check possibleRegion not being null.
> Hbase client throws NoSuchElementException
> ------------------------------------------
>
> Key: HBASE-3878
> URL: https://issues.apache.org/jira/browse/HBASE-3878
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.2
> Reporter: gaojinchao
> Fix For: 0.90.4
>
> Attachments: 3878.patch
>
>
> Soft reference objects, which are cleared at the discretion of the
> garbage collector in response to memory demand.
> I used ycsb to put data and threw exception.
> >>>>
> >>>> Hbase Code:
> >>>> // Cut the cache so that we only get the part that could contain
> >>>> // regions that match our key
> >>>> SoftValueSortedMap<byte[], HRegionLocation> matchingRegions =
> >>>> tableLocations.headMap(row);
> >>>>
> >>>> // if that portion of the map is empty, then we're done. otherwise,
> >>>> // we need to examine the cached location to verify that it is
> >>>> // a match by end key as well.
> >>>> if (!matchingRegions.isEmpty()) {
> >>>> HRegionLocation possibleRegion =
> >>>> matchingRegions.get(matchingRegions.lastKey());
> >>>>
> >>>> ycsb client log:
> >>>>
> >>>> [java] begin StatusThread run
> >>>> [java] java.util.NoSuchElementException
> >>>> [java] at java.util.TreeMap.key(TreeMap.java:1206)
> >>>> [java] at
> >> java.util.TreeMap$NavigableSubMap.lastKey(TreeMap.java:1435)
> >>>> [java] at
> >> org.apache.hadoop.hbase.util.SoftValueSortedMap.lastKey(SoftValueSort
> >> edMap.java:131)
> >>>> [java] at
> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
> >> tation.getCachedLocation(HConnectionManager.java:841)
> >>>> [java] at
> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
> >> tation.locateRegionInMeta(HConnectionManager.java:664)
> >>>> [java] at
> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
> >> tation.locateRegion(HConnectionManager.java:590)
> >>>> [java] at
> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
> >> tation.processBatch(HConnectionManager.java:1114)
> >>>> [java] at
> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
> >> tation.processBatchOfPuts(HConnectionManager.java:1234)
> >>>> [java] at
> >> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
> >>>> [java] at
> >> org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:675)
> >>>> [java] at
> >> org.apache.hadoop.hbase.client.HTable.put(HTable.java:665)
> >>>> [java] at com.yahoo.ycsb.db.HBaseClient.update(Unknown Source)
> >>>> [java] at com.yahoo.ycsb.db.HBaseClient.insert(Unknown Source)
> >>>> [java] at com.yahoo.ycsb.DBWrapper.insert(Unknown Source)
> >>>> [java] at com.yahoo.ycsb.workloads.MyWorkload.doInsert(Unknown
> >> Source)
> >>>> [java] at com.yahoo.ycsb.ClientThread.run(Unknown Source)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Updated] (HBASE-3878) Hbase client throws NoSuchElementException
Posted by Ted Yu <yu...@gmail.com>.
I don't see problem with deleteCachedLocation().
Maybe you were thinking about this line:
tableLocations.remove(rl.getRegionInfo().getStartKey());
Looking at http://www.docjar.com/html/api/java/util/TreeMap.java.html, line
2467, isInRange() is called before the actual removal.
Cheers
On Thu, May 12, 2011 at 6:01 PM, Gaojinchao <ga...@huawei.com> wrote:
> Hi Ted, Thanks for your patch.
>
> I am not familiar with SoftValueSortedMap.
> Please check this function.
>
> void deleteCachedLocation(final byte [] tableName, final byte [] row) {
> synchronized (this.cachedRegionLocations) {
> SoftValueSortedMap<byte [], HRegionLocation> tableLocations =
> getTableLocations(tableName);
> // start to examine the cache. we can only do cache actions
> // if there's something in the cache for this table.
> if (!tableLocations.isEmpty()) {
> HRegionLocation rl = getCachedLocation(tableName, row);
> if (rl != null) {
> tableLocations.remove(rl.getRegionInfo().getStartKey());
> if (LOG.isDebugEnabled()) {
> LOG.debug("Removed " +
> rl.getRegionInfo().getRegionNameAsString() +
> " for tableName=" + Bytes.toString(tableName) +
> " from cache " + "because of " + Bytes.toStringBinary(row));
> }
> }
> }
> }
> }
>
>
> -----邮件原件-----
> 发件人: Ted Yu (JIRA) [mailto:jira@apache.org]
> 发送时间: 2011年5月12日 22:44
> 收件人: Gaojinchao
> 主题: [jira] [Updated] (HBASE-3878) Hbase client throws
> NoSuchElementException
>
>
> [
> https://issues.apache.org/jira/browse/HBASE-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>
> Ted Yu updated HBASE-3878:
> --------------------------
>
> Attachment: 3878.patch
>
> Added try/catch block to guard against NoSuchElementException.
>
> We already check possibleRegion not being null.
>
> > Hbase client throws NoSuchElementException
> > ------------------------------------------
> >
> > Key: HBASE-3878
> > URL: https://issues.apache.org/jira/browse/HBASE-3878
> > Project: HBase
> > Issue Type: Bug
> > Components: client
> > Affects Versions: 0.90.2
> > Reporter: gaojinchao
> > Fix For: 0.90.4
> >
> > Attachments: 3878.patch
> >
> >
> > Soft reference objects, which are cleared at the discretion of the
> > garbage collector in response to memory demand.
> > I used ycsb to put data and threw exception.
> > >>>>
> > >>>> Hbase Code:
> > >>>> // Cut the cache so that we only get the part that could contain
> > >>>> // regions that match our key
> > >>>> SoftValueSortedMap<byte[], HRegionLocation> matchingRegions =
> > >>>> tableLocations.headMap(row);
> > >>>>
> > >>>> // if that portion of the map is empty, then we're done.
> otherwise,
> > >>>> // we need to examine the cached location to verify that it is
> > >>>> // a match by end key as well.
> > >>>> if (!matchingRegions.isEmpty()) {
> > >>>> HRegionLocation possibleRegion =
> > >>>> matchingRegions.get(matchingRegions.lastKey());
> > >>>>
> > >>>> ycsb client log:
> > >>>>
> > >>>> [java] begin StatusThread run
> > >>>> [java] java.util.NoSuchElementException
> > >>>> [java] at java.util.TreeMap.key(TreeMap.java:1206)
> > >>>> [java] at
> > >> java.util.TreeMap$NavigableSubMap.lastKey(TreeMap.java:1435)
> > >>>> [java] at
> > >> org.apache.hadoop.hbase.util.SoftValueSortedMap.lastKey(SoftValueSort
> > >> edMap.java:131)
> > >>>> [java] at
> > >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
> > >> tation.getCachedLocation(HConnectionManager.java:841)
> > >>>> [java] at
> > >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
> > >> tation.locateRegionInMeta(HConnectionManager.java:664)
> > >>>> [java] at
> > >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
> > >> tation.locateRegion(HConnectionManager.java:590)
> > >>>> [java] at
> > >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
> > >> tation.processBatch(HConnectionManager.java:1114)
> > >>>> [java] at
> > >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
> > >> tation.processBatchOfPuts(HConnectionManager.java:1234)
> > >>>> [java] at
> > >> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
> > >>>> [java] at
> > >> org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:675)
> > >>>> [java] at
> > >> org.apache.hadoop.hbase.client.HTable.put(HTable.java:665)
> > >>>> [java] at com.yahoo.ycsb.db.HBaseClient.update(Unknown
> Source)
> > >>>> [java] at com.yahoo.ycsb.db.HBaseClient.insert(Unknown
> Source)
> > >>>> [java] at com.yahoo.ycsb.DBWrapper.insert(Unknown Source)
> > >>>> [java] at
> com.yahoo.ycsb.workloads.MyWorkload.doInsert(Unknown
> > >> Source)
> > >>>> [java] at com.yahoo.ycsb.ClientThread.run(Unknown Source)
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
Re: [jira] [Updated] (HBASE-3878) Hbase client throws
NoSuchElementException
Posted by Gaojinchao <ga...@huawei.com>.
I try to make a patch yesterday. It is same with that by Ted.
I am not sure if the function deleteCachedLocation has some issue because I searched it .
I want to ask for advice. Sorry. I will study it :)
Thanks.
-----邮件原件-----
发件人: Stack [mailto:saint.ack@gmail.com]
发送时间: 2011年5月13日 9:53
收件人: dev@hbase.apache.org
抄送: dev@hbase.apache.org
主题: Re: [jira] [Updated] (HBASE-3878) Hbase client throws NoSuchElementException
What are you suggesting gao? You must be specific about what you would have us look at because we are bad on inference
Thanks
Stack
On May 12, 2011, at 18:01, Gaojinchao <ga...@huawei.com> wrote:
> Hi Ted, Thanks for your patch.
>
> I am not familiar with SoftValueSortedMap.
> Please check this function.
>
> void deleteCachedLocation(final byte [] tableName, final byte [] row) {
> synchronized (this.cachedRegionLocations) {
> SoftValueSortedMap<byte [], HRegionLocation> tableLocations =
> getTableLocations(tableName);
> // start to examine the cache. we can only do cache actions
> // if there's something in the cache for this table.
> if (!tableLocations.isEmpty()) {
> HRegionLocation rl = getCachedLocation(tableName, row);
> if (rl != null) {
> tableLocations.remove(rl.getRegionInfo().getStartKey());
> if (LOG.isDebugEnabled()) {
> LOG.debug("Removed " +
> rl.getRegionInfo().getRegionNameAsString() +
> " for tableName=" + Bytes.toString(tableName) +
> " from cache " + "because of " + Bytes.toStringBinary(row));
> }
> }
> }
> }
> }
>
>
> -----邮件原件-----
> 发件人: Ted Yu (JIRA) [mailto:jira@apache.org]
> 发送时间: 2011年5月12日 22:44
> 收件人: Gaojinchao
> 主题: [jira] [Updated] (HBASE-3878) Hbase client throws NoSuchElementException
>
>
> [ https://issues.apache.org/jira/browse/HBASE-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Ted Yu updated HBASE-3878:
> --------------------------
>
> Attachment: 3878.patch
>
> Added try/catch block to guard against NoSuchElementException.
>
> We already check possibleRegion not being null.
>
>> Hbase client throws NoSuchElementException
>> ------------------------------------------
>>
>> Key: HBASE-3878
>> URL: https://issues.apache.org/jira/browse/HBASE-3878
>> Project: HBase
>> Issue Type: Bug
>> Components: client
>> Affects Versions: 0.90.2
>> Reporter: gaojinchao
>> Fix For: 0.90.4
>>
>> Attachments: 3878.patch
>>
>>
>> Soft reference objects, which are cleared at the discretion of the
>> garbage collector in response to memory demand.
>> I used ycsb to put data and threw exception.
>>>>>>
>>>>>> Hbase Code:
>>>>>> // Cut the cache so that we only get the part that could contain
>>>>>> // regions that match our key
>>>>>> SoftValueSortedMap<byte[], HRegionLocation> matchingRegions =
>>>>>> tableLocations.headMap(row);
>>>>>>
>>>>>> // if that portion of the map is empty, then we're done. otherwise,
>>>>>> // we need to examine the cached location to verify that it is
>>>>>> // a match by end key as well.
>>>>>> if (!matchingRegions.isEmpty()) {
>>>>>> HRegionLocation possibleRegion =
>>>>>> matchingRegions.get(matchingRegions.lastKey());
>>>>>>
>>>>>> ycsb client log:
>>>>>>
>>>>>> [java] begin StatusThread run
>>>>>> [java] java.util.NoSuchElementException
>>>>>> [java] at java.util.TreeMap.key(TreeMap.java:1206)
>>>>>> [java] at
>>>> java.util.TreeMap$NavigableSubMap.lastKey(TreeMap.java:1435)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.util.SoftValueSortedMap.lastKey(SoftValueSort
>>>> edMap.java:131)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
>>>> tation.getCachedLocation(HConnectionManager.java:841)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
>>>> tation.locateRegionInMeta(HConnectionManager.java:664)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
>>>> tation.locateRegion(HConnectionManager.java:590)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
>>>> tation.processBatch(HConnectionManager.java:1114)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
>>>> tation.processBatchOfPuts(HConnectionManager.java:1234)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:675)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HTable.put(HTable.java:665)
>>>>>> [java] at com.yahoo.ycsb.db.HBaseClient.update(Unknown Source)
>>>>>> [java] at com.yahoo.ycsb.db.HBaseClient.insert(Unknown Source)
>>>>>> [java] at com.yahoo.ycsb.DBWrapper.insert(Unknown Source)
>>>>>> [java] at com.yahoo.ycsb.workloads.MyWorkload.doInsert(Unknown
>>>> Source)
>>>>>> [java] at com.yahoo.ycsb.ClientThread.run(Unknown Source)
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Updated] (HBASE-3878) Hbase client throws NoSuchElementException
Posted by Stack <sa...@gmail.com>.
What are you suggesting gao? You must be specific about what you would have us look at because we are bad on inference
Thanks
Stack
On May 12, 2011, at 18:01, Gaojinchao <ga...@huawei.com> wrote:
> Hi Ted, Thanks for your patch.
>
> I am not familiar with SoftValueSortedMap.
> Please check this function.
>
> void deleteCachedLocation(final byte [] tableName, final byte [] row) {
> synchronized (this.cachedRegionLocations) {
> SoftValueSortedMap<byte [], HRegionLocation> tableLocations =
> getTableLocations(tableName);
> // start to examine the cache. we can only do cache actions
> // if there's something in the cache for this table.
> if (!tableLocations.isEmpty()) {
> HRegionLocation rl = getCachedLocation(tableName, row);
> if (rl != null) {
> tableLocations.remove(rl.getRegionInfo().getStartKey());
> if (LOG.isDebugEnabled()) {
> LOG.debug("Removed " +
> rl.getRegionInfo().getRegionNameAsString() +
> " for tableName=" + Bytes.toString(tableName) +
> " from cache " + "because of " + Bytes.toStringBinary(row));
> }
> }
> }
> }
> }
>
>
> -----邮件原件-----
> 发件人: Ted Yu (JIRA) [mailto:jira@apache.org]
> 发送时间: 2011年5月12日 22:44
> 收件人: Gaojinchao
> 主题: [jira] [Updated] (HBASE-3878) Hbase client throws NoSuchElementException
>
>
> [ https://issues.apache.org/jira/browse/HBASE-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Ted Yu updated HBASE-3878:
> --------------------------
>
> Attachment: 3878.patch
>
> Added try/catch block to guard against NoSuchElementException.
>
> We already check possibleRegion not being null.
>
>> Hbase client throws NoSuchElementException
>> ------------------------------------------
>>
>> Key: HBASE-3878
>> URL: https://issues.apache.org/jira/browse/HBASE-3878
>> Project: HBase
>> Issue Type: Bug
>> Components: client
>> Affects Versions: 0.90.2
>> Reporter: gaojinchao
>> Fix For: 0.90.4
>>
>> Attachments: 3878.patch
>>
>>
>> Soft reference objects, which are cleared at the discretion of the
>> garbage collector in response to memory demand.
>> I used ycsb to put data and threw exception.
>>>>>>
>>>>>> Hbase Code:
>>>>>> // Cut the cache so that we only get the part that could contain
>>>>>> // regions that match our key
>>>>>> SoftValueSortedMap<byte[], HRegionLocation> matchingRegions =
>>>>>> tableLocations.headMap(row);
>>>>>>
>>>>>> // if that portion of the map is empty, then we're done. otherwise,
>>>>>> // we need to examine the cached location to verify that it is
>>>>>> // a match by end key as well.
>>>>>> if (!matchingRegions.isEmpty()) {
>>>>>> HRegionLocation possibleRegion =
>>>>>> matchingRegions.get(matchingRegions.lastKey());
>>>>>>
>>>>>> ycsb client log:
>>>>>>
>>>>>> [java] begin StatusThread run
>>>>>> [java] java.util.NoSuchElementException
>>>>>> [java] at java.util.TreeMap.key(TreeMap.java:1206)
>>>>>> [java] at
>>>> java.util.TreeMap$NavigableSubMap.lastKey(TreeMap.java:1435)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.util.SoftValueSortedMap.lastKey(SoftValueSort
>>>> edMap.java:131)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
>>>> tation.getCachedLocation(HConnectionManager.java:841)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
>>>> tation.locateRegionInMeta(HConnectionManager.java:664)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
>>>> tation.locateRegion(HConnectionManager.java:590)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
>>>> tation.processBatch(HConnectionManager.java:1114)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplemen
>>>> tation.processBatchOfPuts(HConnectionManager.java:1234)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:675)
>>>>>> [java] at
>>>> org.apache.hadoop.hbase.client.HTable.put(HTable.java:665)
>>>>>> [java] at com.yahoo.ycsb.db.HBaseClient.update(Unknown Source)
>>>>>> [java] at com.yahoo.ycsb.db.HBaseClient.insert(Unknown Source)
>>>>>> [java] at com.yahoo.ycsb.DBWrapper.insert(Unknown Source)
>>>>>> [java] at com.yahoo.ycsb.workloads.MyWorkload.doInsert(Unknown
>>>> Source)
>>>>>> [java] at com.yahoo.ycsb.ClientThread.run(Unknown Source)
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see: http://www.atlassian.com/software/jira