You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jieshan Bean (Created) (JIRA)" <ji...@apache.org> on 2011/12/22 05:19:30 UTC
[jira] [Created] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
A concurrency issue on SoftValueSortedMap
-----------------------------------------
Key: HBASE-5088
URL: https://issues.apache.org/jira/browse/HBASE-5088
Project: HBase
Issue Type: Bug
Components: client
Affects Versions: 0.90.4, 0.94.0
Reporter: Jieshan Bean
SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
But in HConnectionManager#getCachedLocation, it use heapMap to get a view from SoftValueSortedMap#internalMap. Once we operate
on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181900#comment-13181900 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
It seems my reply is too late. Thank you all:)
I suggest backport this patch to 0.90.6.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-final3.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Attachment: (was: 5088.generics.txt)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181663#comment-13181663 ]
Zhihong Yu commented on HBASE-5088:
-----------------------------------
That makes sense.
Some lines are too wide:
{code}
+ return new SoftValueSortedMap<K,V>(this.internalMap.subMap(fromKey, toKey), sync);
{code}
{code}
- SoftValueSortedMap<byte[], HRegionLocation> tableLocs =
- this.cachedRegionLocations.get(key);
+ Map<byte[], HRegionLocation> tableLocs = this.cachedRegionLocations
+ .get(key);
{code}
Did you get the above through auto-formatting ? I think keeping this.cachedRegionLocations.get(key) on one line is better.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181175#comment-13181175 ]
Hadoop QA commented on HBASE-5088:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12509670/HBase5088-trunk-replaceTreeMap.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javadoc. The javadoc tool appears to have generated -151 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 79 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.master.TestSplitLogManager
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/683//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/683//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/683//console
This message is automatically generated.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181915#comment-13181915 ]
ramkrishna.s.vasudevan commented on HBASE-5088:
-----------------------------------------------
@Lars
I too feel we can backport to 0.90?
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-final3.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181870#comment-13181870 ]
Hudson commented on HBASE-5088:
-------------------------------
Integrated in HBase-0.92 #229 (See [https://builds.apache.org/job/HBase-0.92/229/])
HBASE-5088 A concurrency issue on SoftValueSortedMap (Jieshan Bean and Lars H)
larsh :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/SoftValueSortedMap.java
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-final3.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Status: Open (was: Patch Available)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180256#comment-13180256 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
We can finish the tests tommorow,then i'll give the results. Thank you, Lars:)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181704#comment-13181704 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
I assume this should go into 0.90 as well.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Attachment: 5088-final2.txt
How about this one.
* Undid the auto-format that put .get in a separate line.
* broke up line that is too long
* added java doc to new constructor
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181529#comment-13181529 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
@Ted: This change is the minimum we *must* do (in the sense that it is the minimum extra synchronization to make it correct), even it takes a perf hit.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183980#comment-13183980 ]
Hudson commented on HBASE-5088:
-------------------------------
Integrated in HBase-TRUNK #2617 (See [https://builds.apache.org/job/HBase-TRUNK/2617/])
HBASE-5088 addendum
HBASE-5088 A concurrency issue on SoftValueSortedMap (Jieshan Bean and Lars H)
larsh :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/SoftValueSortedMap.java
larsh :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/SoftValueSortedMap.java
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0, 0.90.6
>
> Attachments: 5088-0.90.txt, 5088-0.92-trunk-addendum.txt, 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Attachment: 5088.generics.txt
Here's a version that does that.
o SoftValueSortedMap implements NavigableMap
o All extra NavigableMap are delegated to internalMap
o None of its methods are synchronized
o All generics warnings are fixed.
Please have a look.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181656#comment-13181656 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
Look at the {head|tail|sub}Map methods, that's where it gets interesting and is exactly the reason why there is an extra object to synchronize on.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Attachment: (was: 5088-syncObj.txt)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-final3.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175142#comment-13175142 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
If the observed behavior is for the calling thread to enter an infinite loop (as Jieshan seems to suggest) I'd say this is a critical issue. (imagine this happening in a long lived application server)
Jieshan, can confirm that what you saw was a thread entering looping indefinitely?
As I said above, I think using SoftReferences for caching is generally misguided.
As far as I am concerned we can just replace SoftValueSortedMap with ConcurrentSkipListMap.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180223#comment-13180223 ]
Hadoop QA commented on HBASE-5088:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12508581/5088.generics.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javadoc. The javadoc tool appears to have generated -151 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 79 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestSplitLogManager
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestImportTsv
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/671//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/671//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/671//console
This message is automatically generated.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182175#comment-13182175 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
Removed some of the attachment that are just confusing. The addendum also fixes minor formatting inconsistencies that I had introduced. All is good now.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0, 0.90.6
>
> Attachments: 5088-0.90.txt, 5088-0.92-trunk-addendum.txt, 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-5088:
--------------------------------
Attachment: HBase-5088-trunk.patch
HBase-5088-90.patch
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Attachment: 5088-final.txt
Here's a proposal for a final patch.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175498#comment-13175498 ]
Zhihong Yu commented on HBASE-5088:
-----------------------------------
First, I wonder why this problem wasn't discovered earlier.
@Jieshan:
What JDK version is used ? Please also give us the information on OS as well.
Second I want to see the breakdown of slowdown across reads vs writes.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181515#comment-13181515 ]
Lars Hofhansl edited comment on HBASE-5088 at 1/6/12 7:00 PM:
--------------------------------------------------------------
I think we should check in the latest change. Since we cannot reliably reproduce the issue, we have to verify by inspection. The latest change (5088-syncObject.txt) will be least amount of change from the original code and adds the least extra synchronization needed.
I'll fix up the patch slightly (pull in the interface cleanups from my other patch).
Ted, Stack, etc, please weigh in.
was (Author: lhofhansl):
I think we should check in the latest change. Since we cannot reliably reproduce the issue, we have verify by inspection. The latest change (5088-syncObject.txt) will be least amount of change from the original code and adds the lowest least extra synchronization needed.
I'll fix up the patch slightly (pull in the interface cleanups from my other patch).
Ted, Stack, etc, please weigh in.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Anoop Sam John (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174613#comment-13174613 ]
Anoop Sam John commented on HBASE-5088:
---------------------------------------
Can we use a thread safe impl of the SortedMap ........ ConcurrentSkipListMap?
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use heapMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl resolved HBASE-5088.
----------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Committed to 0.92 and trunk.
Not entirely convinced we should make this change in 0.90
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-final3.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-5088:
--------------------------------
Attachment: (was: HBase-5088-trunk.patch)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181693#comment-13181693 ]
Zhihong Yu commented on HBASE-5088:
-----------------------------------
final2 ? That's funny :-)
This looks good.
+1
Minor: long line still:
{code}
+ Map<byte[], HRegionLocation> tableLocations = getTableLocations(tableName);
{code}
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13185112#comment-13185112 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
Thanks for tracking down the problem Jieshan.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0, 0.90.6
>
> Attachments: 5088-0.90.txt, 5088-0.92-trunk-addendum.txt, 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182204#comment-13182204 ]
Hudson commented on HBASE-5088:
-------------------------------
Integrated in HBase-TRUNK-security #67 (See [https://builds.apache.org/job/HBase-TRUNK-security/67/])
HBASE-5088 addendum
larsh :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/SoftValueSortedMap.java
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0, 0.90.6
>
> Attachments: 5088-0.90.txt, 5088-0.92-trunk-addendum.txt, 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181524#comment-13181524 ]
Zhihong Yu commented on HBASE-5088:
-----------------------------------
I think we'd better verify the performance first.
This issue is not so critical that 0.92 RC3 should be blocked since it is not easily reproducible.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175270#comment-13175270 ]
stack commented on HBASE-5088:
------------------------------
@Lars Sounds bad. Not enough to kill RC IMO since its been this way so long but should make next RC if one and backport to 0.90.6 I'd say.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Attachment: 5088-syncObj.txt
Here's a patch for sync object approach. It uses the internalMap itself to synchronize. When a new SoftValueSortedMap is created via {head|tail|sub}Map the parent's sync object is passed to the newly created map.
This way to more memory is wasted as the {head|tail|sub}map hold a reference to their parents anyway.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13178295#comment-13178295 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
@Jieshan: What do you think of my attached 5088.generics.txt patch?
It switches TreeMap with ConcurrentSkipListMap and also:
* fixes are generics warnings
* uses proper interface instead of concrete classes
* remove all synchronization (should not be necessary with the inner map being ConcurrentSkipListMap).
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176074#comment-13176074 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
@Ted:
Upgrate the jdk version seems very difficult to us, because all the applications base on that version. I remember the 1.6.0_22 is the recommanded version from community before. Can you tell me why should I use "1.6.0_29"? thanks.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175306#comment-13175306 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
I have made a patch replace the TreeMap with ConcurrentSplitListMap in SoftValueSortedMap, but not replace SoftValueSortedMap. I'm running the test on that patch. once finished, I'll submit the patch.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179843#comment-13179843 ]
stack commented on HBASE-5088:
------------------------------
+1 on last patch posted by Lars.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-5088:
--------------------------------
Description:
SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
on this view map(like add/delete) in other threads, a concurrency issue may occur.
was:
SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
But in HConnectionManager#getCachedLocation, it use heapMap to get a view from SoftValueSortedMap#internalMap. Once we operate
on this view map(like add/delete) in other threads, a concurrency issue may occur.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175975#comment-13175975 ]
Zhihong Yu commented on HBASE-5088:
-----------------------------------
@Jieshan:
Can you use a more recent JVM when performing future performance evaluation ?
Such as:
java version "1.6.0_29"
Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-10M3527)
Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175554#comment-13175554 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
I was worrying too why have not discovered that sooner. I assume it just happens rarely and since it leads to an infinite loop rather than an exception people might just miss it.
Jieshan, I was wondering whether you do another comparison with SoftvalueSortedMap replaced by ConcurrentSkiplistMap.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Status: Patch Available (was: Open)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182217#comment-13182217 ]
Hudson commented on HBASE-5088:
-------------------------------
Integrated in HBase-0.92-security #67 (See [https://builds.apache.org/job/HBase-0.92-security/67/])
HBASE-5088 addendum
larsh :
Files :
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/SoftValueSortedMap.java
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0, 0.90.6
>
> Attachments: 5088-0.90.txt, 5088-0.92-trunk-addendum.txt, 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181217#comment-13181217 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
If the new instance of SoftValueSortedMap hold a reference of "sync", then the problem can also be solved. It's a good approach:)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-5088:
--------------------------------
Attachment: (was: HBase5088-trunk-replaceTreeMap.patch)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181864#comment-13181864 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
I made an 0.90 patch, in case somebody wants this in 0.90 as well.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-final3.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175565#comment-13175565 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
Looking at trunk patch... Looks good.
Since the internalMap (which should really be called delegate, but that is a different story) is now a ConcurrentNavigableMap, SoftValueSortedMap could implement ConcurrentNavigableMap and delegate all extra methods to the internalMap. If that is done then more concrete Map usages in HConnectionManager could be replaced by interfaces.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-5088:
--------------------------------
Attachment: PerformanceTestResults.png
HBase5088-trunk-replaceTreeMap.patch
HBase5088-90-replaceTreeMap.patch
HBase5088-90-replaceSoftValueSortedMap.patch
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181632#comment-13181632 ]
Hadoop QA commented on HBASE-5088:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12509707/5088-final.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javadoc. The javadoc tool appears to have generated -151 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 79 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/690//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/690//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/690//console
This message is automatically generated.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182199#comment-13182199 ]
ramkrishna.s.vasudevan commented on HBASE-5088:
-----------------------------------------------
@Lars
Good on you for committing it to 0.90 :)..
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0, 0.90.6
>
> Attachments: 5088-0.90.txt, 5088-0.92-trunk-addendum.txt, 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182192#comment-13182192 ]
Hudson commented on HBASE-5088:
-------------------------------
Integrated in HBase-0.92 #235 (See [https://builds.apache.org/job/HBase-0.92/235/])
HBASE-5088 addendum
larsh :
Files :
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/SoftValueSortedMap.java
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0, 0.90.6
>
> Attachments: 5088-0.90.txt, 5088-0.92-trunk-addendum.txt, 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181156#comment-13181156 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
Either the patch of replace TreeMap with ConcurrentSkipListMap, or the patch of replace SoftValueSortedMap with ConcurrentSkipListMap, the performance slightly degraded. The latter one seems better. Please find the test results from the attachment "PerformanceTestResults.png".
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean reassigned HBASE-5088:
-----------------------------------
Assignee: Jieshan Bean
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use heapMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl reassigned HBASE-5088:
------------------------------------
Assignee: Lars Hofhansl (was: Jieshan Bean)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175274#comment-13175274 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
Agree... Shouldn't kill the RC.
One thing I noticed in the code, we always use the concrete Map type everywhere. This is in general bad design. Should only the interface that is needed.
Most methods only use the Map interface. Shouldn't be passed anything more then.
I can attach a patch the reduces the Map interface used to what is actually needed (that may or may not make this problem simpler, but it'll be nicer ;) ).
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174606#comment-13174606 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
We found this issue while one thread can't get out of the loop in TreeMap#fixAfterDeletion:
{noformat}
"Thread-923" prio=10 tid=0x00007f3d40553000 nid=0x3ed6 runnable [0x00007f3d05c1a000]
java.lang.Thread.State: RUNNABLE
at java.util.TreeMap.fixAfterDeletion(TreeMap.java:2193)
at java.util.TreeMap.deleteEntry(TreeMap.java:2151)
at java.util.TreeMap.remove(TreeMap.java:585)
at java.util.TreeMap$NavigableSubMap.remove(TreeMap.java:1395)
at org.apache.hadoop.hbase.util.SoftValueSortedMap.get(SoftValueSortedMap.java:101)
- locked <0x00007f3d94f24f70> (a org.apache.hadoop.hbase.util.SoftValueSortedMap)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:846)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:668)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:559)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1018)
at org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at com.huawei.icbc.query.SingleTabQuery.querybatch(SingleTabQuery.java:197)
at com.huawei.icbc.benchmark.SingleTabQueryAction.query(SingleTabQueryAction.java:181)
at framework.QueryThread.run(QueryThread.java:47)
{noformat}
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use heapMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175412#comment-13175412 ]
ramkrishna.s.vasudevan commented on HBASE-5088:
-----------------------------------------------
+1 on patch..
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181558#comment-13181558 ]
Zhihong Yu commented on HBASE-5088:
-----------------------------------
@Lars:
Agreed.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175253#comment-13175253 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
Loop into the mothod of TreeMap#fixAfterDeletion(Entry<K,V> x), once the x is null(It may caused by a concurrency issue), it can't come out of that loop. At that time, CPU usage is high.
That's what we saw. All the time, the thread was blocked in that method.
Currently, the patch is made just replace TreeMap with ConcurrentSkipListMap as Anoop and Lars's suggestion. We're verifying the patch.
@Ted, actually, the heapmap method of ConcurrentSkipListMap is different from TreeMap, it is also backed by the original thread-safe map. what do you think?
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175257#comment-13175257 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
@Stack... Do you think this is critical? (it has been like this forever, so maybe it's not, seems bad enough, though)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181178#comment-13181178 ]
Lars Hofhansl edited comment on HBASE-5088 at 1/6/12 7:24 AM:
--------------------------------------------------------------
Here's a patch for sync object approach. It uses the internalMap itself to synchronize. When a new SoftValueSortedMap is created via {head|tail|sub}Map the parent's sync object is passed to the newly created map.
This way no memory is wasted as the {head|tail|sub}map hold a reference to their parents anyway.
was (Author: lhofhansl):
Here's a patch for sync object approach. It uses the internalMap itself to synchronize. When a new SoftValueSortedMap is created via {head|tail|sub}Map the parent's sync object is passed to the newly created map.
This way to more memory is wasted as the {head|tail|sub}map hold a reference to their parents anyway.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181884#comment-13181884 ]
Hudson commented on HBASE-5088:
-------------------------------
Integrated in HBase-TRUNK-security #66 (See [https://builds.apache.org/job/HBase-TRUNK-security/66/])
HBASE-5088 A concurrency issue on SoftValueSortedMap (Jieshan Bean and Lars H)
larsh :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/SoftValueSortedMap.java
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-final3.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174659#comment-13174659 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
SoftValueSortedMap is (currently) only used in HConnectionImplementation, so that should be OK.
It seems that ReferenceQueue is thread safe and hence we won't need to synchronize any of the methods in SoftValueSortedMap.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Status: Open (was: Patch Available)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5088:
-------------------------
Priority: Critical (was: Major)
Fix Version/s: 0.92.0
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176083#comment-13176083 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
Thanks for your time on this Jieshan!
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181161#comment-13181161 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
@Lars:
"patch of replace TreeMap with ConcurrentSkipListMap" is same with the patch you attached named "5088.generics.txt".
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181192#comment-13181192 ]
Hadoop QA commented on HBASE-5088:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12509671/5088-syncObj.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javadoc. The javadoc tool appears to have generated -151 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 79 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/684//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/684//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/684//console
This message is automatically generated.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181602#comment-13181602 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
So I take this as a +1?
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175492#comment-13175492 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
probably because of ConcurrenSkiplistMap... it is definitely slower than a plain TreeMap.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175256#comment-13175256 ]
Zhihong Yu commented on HBASE-5088:
-----------------------------------
I think Anoop and Lars' suggestion is good.
Let us know the result of your tests.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175640#comment-13175640 ]
ramkrishna.s.vasudevan commented on HBASE-5088:
-----------------------------------------------
@Lars
Had a look at the patch. The intention is good.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181647#comment-13181647 ]
Zhihong Yu commented on HBASE-5088:
-----------------------------------
{code}
+ private SoftValueSortedMap(SortedMap<K,SoftValue<K,V>> original, Object sync) {
{code}
The new ctor is private. Meaning, sync would always be the same as original. I wonder if introducing the sync field is necessary.
Can this:
{code}
+ synchronized(sync) {
{code}
be replaced by the following ?
{code}
+ synchronized(this.internalMap) {
{code}
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Fix Version/s: 0.90.6
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0, 0.90.6
>
> Attachments: 5088-0.90.txt, 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Attachment: 5088-final3.txt
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-final3.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181911#comment-13181911 ]
Hudson commented on HBASE-5088:
-------------------------------
Integrated in HBase-0.92-security #65 (See [https://builds.apache.org/job/HBase-0.92-security/65/])
HBASE-5088 A concurrency issue on SoftValueSortedMap (Jieshan Bean and Lars H)
larsh :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/SoftValueSortedMap.java
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-final3.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174646#comment-13174646 ]
Lars Hofhansl edited comment on HBASE-5088 at 12/22/11 6:35 AM:
----------------------------------------------------------------
ConcurrentSkipListMap does not appear to have this problem as a {head|tail}Map just maps put/get operations back to the original map. Maybe we can do the same approach here.
Cannot use ConcurrentSkipListMap directly as SoftValueSortedMap specifically stores SoftReferences to the values... Although I am personally not a big fan of this. Either we cache stuff or not. If there is memory pressure and the JVM removes the object referred to by the soft references, we generate even more new garbage when we miss the cache and have to reacquire the object.
Looks like SoftValueSortedMap.headMap() is buggy, we should throw UnsupportedOperation or fix it. Fixing it would mean to keep a reference to the original map and use that synchronize in the headMap (which means we need to return a special inner class), or to materialize the headmap in a copy, or do what ConcurrentSkipListMap does and map the API operations back to the original map.
was (Author: lhofhansl):
ConcurrentSkipListMap does not appear to have this problem as a {head|tail}Map just maps put/get operations back to the original map. Maybe we can do the same approach here.
Cannot use ConcurrentSkipListMap as SoftValueSortedMap specifically stores SoftReferences to the values... Although I am personally not a big fan of this. Either we cache stuff or not. If there is memory pressure and the JVM removes the object referred to by the soft references, we generate even more new garbage when we miss the cache and have to reacquire the object.
Looks like SoftValueSortedMap.headMap() is buggy, we should throw UnsupportedOperation or fix it. Fixing it would mean to keep a reference to the original map and use that synchronize in the headMap (which means we need to return a special inner class), or to materialize the headmap in a copy, or do what ConcurrentSkipListMap does and map the API operations back to the original map.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175303#comment-13175303 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
I also have a version that replaces SoftValueSortedMap with ConcurrentSkipListMap.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181515#comment-13181515 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
I think we should check in the latest change. Since we cannot reliably reproduce the issue, we have verify by inspection. The latest change (5088-syncObject.txt) will be least amount of change from the original code and adds the lowest least extra synchronization needed.
I'll fix up the patch slightly (pull in the interface cleanups from my other patch).
Ted, Stack, etc, please weigh in.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174620#comment-13174620 ]
Zhihong Yu commented on HBASE-5088:
-----------------------------------
I was looking at http://fuseyism.com/classpath/doc/java/util/concurrent/ConcurrentSkipListMap-source.html and trying to see whether its headMap() implementation is different from that of SoftValueSortedMap.
I suggest trying option #1 above first.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use heapMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181695#comment-13181695 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
I thought so too :)
That line went through the formatter (just checked again) (84 chars)
I can break it up anyway, will do at commit.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-5088:
--------------------------------
Attachment: (was: HBase5088-90-replaceTreeMap.patch)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Attachment: (was: 5088-final2.txt)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182046#comment-13182046 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
OK. Sorry for presuming. I'll commit to 0.90 later today.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-final3.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181637#comment-13181637 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
In a quick local test (on pseudo distributed mode) I was able to reproduce the slow down with ConcurrentSkipListMap that Jieshan has reported, in my case the slowdown was even worse (runtime went up from ~12.5s to ~17.5 - everything is faster in local mode, so a constant slowdown will have a larger proportional impact).
With the last patch (sync Object) I have not seen any slowdown using the same test. So I think this is good to go a soon as HadoopQA confirms.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176075#comment-13176075 ]
Zhihong Yu commented on HBASE-5088:
-----------------------------------
There was no particular reason.
Please go ahead with planned comparison.
At the same time, I am polling concurrency user group for experts' comments.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181465#comment-13181465 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
Do you think you can run your test with this again for comparison?
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175887#comment-13175887 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
@Ted,
At first, I also thought we would get a higher performance with this patch, because all the keywords of "synchronized" removed. But it slowdown.
I agree with the explaination from Lars.
Our JDK version is 1.6.0_22. And the below is our OS information:
{noformat}
C3S3:~ # cat /proc/version
Linux version 2.6.32.12-0.7-default (geeko@buildhost) (gcc version 4.3.4 [gcc-4_ 3-branch revision 152973] (SUSE Linux) ) #1 SMP 2010-05-20 11:14:20 +0200
C3S3:/proc # lsb_release -a
LSB Version: core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64:desktop-4.0-amd64:desktop-4.0-noarch:graphics-2.0-amd64:graphics-2.0-noarch:graphics-3.2-amd64:graphics-3.2-noarch:graphics-4.0-amd64:graphics-4.0-noarch
Distributor ID: SUSE LINUX
Description: SUSE Linux Enterprise Server 11 (x86_64)
Release: 11
Codename: n/a
{noformat}
We'll take more tests accross the read vs write and give out the results.
@Lars,
Sorry, I didn't do another comparison with SoftvalueSortedMap replaced by ConcurrentSkiplistMap.And am planning to do it. Including the functional test and the performance test. And then, we can choose a better one.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175283#comment-13175283 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
+1
Use the concrete Map type is always not recommended.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Fix Version/s: (was: 0.90.6)
0.94.0
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-final3.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181159#comment-13181159 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
I assume that "patch of replace TreeMap with ConcurrentSkipListMap" is the version that removed all synchronization from SoftValueSortedMap?
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5088:
------------------------------------------
Fix Version/s: 0.90.6
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-5088:
--------------------------------
Attachment: HBase5088-trunk-replaceTreeMap.patch
HBase5088-90-replaceTreeMap.patch
HBase5088-90-replaceSoftValueSortedMap.patch
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174623#comment-13174623 ]
ramkrishna.s.vasudevan commented on HBASE-5088:
-----------------------------------------------
{code}
"WriterThread-2" prio=10 tid=0x00000000405cb000 nid=0x6a98 runnable [0x00007fd0c18cf000]
java.lang.Thread.State: RUNNABLE
at org.apache.hadoop.hbase.util.Bytes.compareTo(Bytes.java:916)
at org.apache.hadoop.hbase.util.Bytes$ByteArrayComparator.compare(Bytes.java:109)
at org.apache.hadoop.hbase.util.Bytes$ByteArrayComparator.compare(Bytes.java:101)
at java.util.TreeMap.getEntryUsingComparator(TreeMap.java:351)
at java.util.TreeMap.getEntry(TreeMap.java:322)
at java.util.TreeMap.remove(TreeMap.java:580)
at org.apache.hadoop.hbase.util.SoftValueSortedMap.checkReferences(SoftValueSortedMap.java:76)
at org.apache.hadoop.hbase.util.SoftValueSortedMap.isEmpty(SoftValueSortedMap.java:150)
- locked <0x00007fd106cd6870> (a org.apache.hadoop.hbase.util.SoftValueSortedMap)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:826)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:668)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1133)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1253)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:828)
at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:684)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:674)
at PerRegionLoadData$WriterThread.writeBuffer(PerRegionLoadData.java:350)
at PerRegionLoadData$WriterThread.doRun(PerRegionLoadData.java:314)
{code}
Once we got the above stack trace.. did not dig in much that time.. good finding
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use heapMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181657#comment-13181657 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
But I see how this is easy to miss, maybe some comments on that constructor would be in order.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Attachment: 5088-0.92-trunk-addendum.txt
This is the addendum. Now the code is pretty clean w.r.t. using Map interfaces.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0, 0.90.6
>
> Attachments: 5088-0.90.txt, 5088-0.92-trunk-addendum.txt, 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175488#comment-13175488 ]
Zhihong Yu commented on HBASE-5088:
-----------------------------------
@Jieshan:
What is the unit of comparison you gave @ 23/Dec/11 12:25 ?
If possible, can you get numbers for read and write, respectively.
{code}
- public synchronized V put(K key, V value) {
+ public V put(K key, V value) {
{code}
One might think higher concurrency should be achived by removing synchronized keyword.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Attachment: 5088-0.90.txt
This is what I committed to 0.90.
While I worked on that I noticed that I can get rid more concrete uses of SoftvalueSortedMap in HConnectionManager (in fact all uses except creation, which is nice). I'll do an addendum for this in 0.92 and trunk.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-0.90.txt, 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-5088:
--------------------------------
Attachment: HBase5088Reproduce.java
HBase-5088-trunk.patch
HBase-5088-90.patch
This problem can't be easily reproduced in the cluster. So I just mocked it by write some test code to reproduce this same concurrency issue. See HBase5088Reproduce.java, it can be reproduced with a high probability. Change the TreeMap to ConcurrentSkipListMap can solve the problem.
The patches, I have tested them. All the unit tests passed. In cluster, the performance seems a slighly coming down:
38764-without patch and 37080-with patch
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181164#comment-13181164 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
Thanks!
Some performance hit was expected, as we didn't handle concurrency correctly before.
But, wow 10% degradation, didn't expect ConcurrentSkipListMap to have such an effect.
Hmm... Curious, since this is a client only change, did you happen to notice higher (or lower) CPU usage on the client?
The current code is clearly not correct. Now I am wondering whether instances of SoftValueSortedMap should be passed an optional "synchronizer object". By default it would be "this", but for instances created from headMap, etc, it would be the "synchronizer object" of the parent. Then instead of all methods marked as synchronized they'd all synchronize on the "synchronizer object".
Do folks think this would be reasonable approach to validate?
I also would like to get opinions about doing away with SoftValueSortedMap and using ConcurrentSkipListMap directly (~5% performance hit, and it would no longer allow the JVM to remove the cached values under memory pressure - which I think never works correctly anyway).
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181712#comment-13181712 ]
Hadoop QA commented on HBASE-5088:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12509738/5088-final2.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javadoc. The javadoc tool appears to have generated -151 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 79 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestImportTsv
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/691//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/691//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/691//console
This message is automatically generated.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-final3.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Attachment: (was: 5088-useMapInterfaces.txt)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final.txt, 5088-final2.txt, 5088-final3.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180225#comment-13180225 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
These four test frequently (always?) fail.
Would be nice to get this in before 0.92.
Since this ends up in an infinite loop, it is hard to replicate in a test.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Attachment: (was: 5088-final.txt)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 5088-final3.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181608#comment-13181608 ]
Zhihong Yu commented on HBASE-5088:
-----------------------------------
I haven't looked at the final patch yet - can do that later.
So consider my vote +0.
Please run the patch through Hadoop QA.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0, 0.90.6
>
> Attachments: 5088-final.txt, 5088-syncObj.txt, 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-90-replaceSoftValueSortedMap.patch, HBase5088-90-replaceTreeMap.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Anoop Sam John (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174653#comment-13174653 ]
Anoop Sam John commented on HBASE-5088:
---------------------------------------
@Lars I mean replace the TreeMap in SoftValueSortedMap with ConcurrentSkipListMap.
Yes we need the SoftValueSortedMap as it maintains Soft refs...
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Attachment: 5088-useMapInterfaces.txt
Trivial patch to use the least specific Map interface possible in HConnectionManager.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175309#comment-13175309 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
Thanks Lars, I will modify the patch under your suggestion, to use the interface replace the concrete type:)...Will submit it later.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-5088:
--------------------------------
Attachment: (was: HBase5088-90-replaceSoftValueSortedMap.patch)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088-trunk-replaceTreeMap.patch, HBase5088Reproduce.java, PerformanceTestResults.png
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174646#comment-13174646 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
ConcurrentSkipListMap does not appear to have this problem as a {head|tail}Map just maps put/get operations back to the original map. Maybe we can do the same approach here.
Cannot use ConcurrentSkipListMap as SoftValueSortedMap specifically stores SoftReferences to the values... Although I am personally not a big fan of this. Either we cache stuff or not. If there is memory pressure and the JVM removes the object referred to by the soft references, we generate even more new garbage when we miss the cache and have to reacquire the object.
Looks like SoftValueSortedMap.headMap() is buggy, we should throw UnsupportedOperation or fix it. Fixing it would mean to keep a reference to the original map and use that synchronize in the headMap (which means we need to return a special inner class), or to materialize the headmap in a copy, or do what ConcurrentSkipListMap does and map the API operations back to the original map.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5088:
---------------------------------
Status: Patch Available (was: Open)
Getting a test run in.
Jieshan, I will wait for your OK/testresults on the patch before committing.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13178295#comment-13178295 ]
Lars Hofhansl edited comment on HBASE-5088 at 1/2/12 6:04 AM:
--------------------------------------------------------------
@Jieshan: What do you think of my attached 5088.generics.txt patch?
It switches TreeMap with ConcurrentSkipListMap and also:
* fixes all generics warnings
* uses proper interfaces instead of concrete classes
* removes all synchronization (should not be necessary with the inner map being a ConcurrentSkipListMap).
was (Author: lhofhansl):
@Jieshan: What do you think of my attached 5088.generics.txt patch?
It switches TreeMap with ConcurrentSkipListMap and also:
* fixes are generics warnings
* uses proper interface instead of concrete classes
* remove all synchronization (should not be necessary with the inner map being ConcurrentSkipListMap).
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176078#comment-13176078 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
ok..Thanks...We're doing the performance tests accross reads vs writes with two different patches according to your suggestion:)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175632#comment-13175632 ]
Lars Hofhansl edited comment on HBASE-5088 at 12/24/11 12:58 AM:
-----------------------------------------------------------------
Here's a version that does that.
o SoftValueSortedMap implements NavigableMap
o All extra NavigableMap methods are delegated to internalMap
o None of its methods are synchronized
o All generics warnings are fixed.
Please have a look.
was (Author: lhofhansl):
Here's a version that does that.
o SoftValueSortedMap implements NavigableMap
o All extra NavigableMap are delegated to internalMap
o None of its methods are synchronized
o All generics warnings are fixed.
Please have a look.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Anoop Sam John (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174612#comment-13174612 ]
Anoop Sam John commented on HBASE-5088:
---------------------------------------
Good catch Jieshan
SoftValueSortedMap.headMap () returns a new Object of SoftValueSortedMap.
This new object's internalMap is a view of the original internalMap which is refered by the 1st SoftValueSortedMap.. But the view always refer and operates on the actual TreeMap datastructure which is not Thread safe....
Now 2 threads can operate on the 2 SoftValueSortedMap objects concurrently which will result in concurrent operation on a single TreeMap object..
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use heapMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jieshan Bean updated HBASE-5088:
--------------------------------
Attachment: (was: HBase-5088-90.patch)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13178362#comment-13178362 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
@Lars:
It makes sense to me. I also made plan to test this patch, but the tests haven't finished yet(Encountered some problems during the tests).
Thank you for your attention on this issue, Lars:)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
> Attachments: 5088-useMapInterfaces.txt, 5088.generics.txt, HBase-5088-90.patch, HBase-5088-trunk.patch, HBase5088Reproduce.java
>
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Jieshan Bean (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174615#comment-13174615 ]
Jieshan Bean commented on HBASE-5088:
-------------------------------------
Good suggestion, Anoop.
Actually there's 2 solutions now:
1. Use "synchronized" while we operation on the view of the original map.
2. Use ConcurrentSkipListMap insteadof TreeMap.
Don't know which one is better.
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use heapMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5088) A concurrency issue on
SoftValueSortedMap
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174652#comment-13174652 ]
Lars Hofhansl commented on HBASE-5088:
--------------------------------------
Or maybe I misunderstood and you want just replace the TreeMap in SoftValueSortedMap with ConcurrentSkipListMap... That would work too :)
> A concurrency issue on SoftValueSortedMap
> -----------------------------------------
>
> Key: HBASE-5088
> URL: https://issues.apache.org/jira/browse/HBASE-5088
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.94.0
> Reporter: Jieshan Bean
> Assignee: Jieshan Bean
>
> SoftValueSortedMap is backed by a TreeMap. All the methods in this class are synchronized. If we use this method to add/delete elements, it's ok.
> But in HConnectionManager#getCachedLocation, it use headMap to get a view from SoftValueSortedMap#internalMap. Once we operate
> on this view map(like add/delete) in other threads, a concurrency issue may occur.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira