You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jonathan Gray (Created) (JIRA)" <ji...@apache.org> on 2011/10/14 08:04:11 UTC
[jira] [Created] (HBASE-4589) CacheOnWrite broken in some cases
because it can conflict with evictOnClose
CacheOnWrite broken in some cases because it can conflict with evictOnClose
---------------------------------------------------------------------------
Key: HBASE-4589
URL: https://issues.apache.org/jira/browse/HBASE-4589
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.92.0, 0.94.0
Reporter: Jonathan Gray
Assignee: Jonathan Gray
Priority: Critical
Fix For: 0.92.0
Commit of HBASE-4078 added some extra StoreFile verification which just did an open of a StoreFile reader and then closes it, ensuring there's no exception. If evict-on-close is on, which it is by default, this causes all blocks of a file to be evicted even though it's still open.
We need to add the boolean into the close call in the way we have booleans for cacheBlocks at some point since we need to make localized decisions in some cases.
In lots of places, we can always rely on cacheConf.shouldEvictOnClose() so shouldn't be too burdensome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4589) CacheOnWrite broken in some cases
because it can conflict with evictOnClose
Posted by "Jonathan Gray (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-4589:
---------------------------------
Resolution: Fixed
Release Note: EvictOnClose can be turned on/off on a per-file-close basis
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to 92 branch and trunk. Thanks stack.
> CacheOnWrite broken in some cases because it can conflict with evictOnClose
> ---------------------------------------------------------------------------
>
> Key: HBASE-4589
> URL: https://issues.apache.org/jira/browse/HBASE-4589
> Project: HBase
> Issue Type: Bug
> Components: io
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: HBASE-4589-v1.patch
>
>
> Commit of HBASE-4078 added some extra StoreFile verification which just did an open of a StoreFile reader and then closes it, ensuring there's no exception. If evict-on-close is on, which it is by default, this causes all blocks of a file to be evicted even though it's still open.
> We need to add the boolean into the close call in the way we have booleans for cacheBlocks at some point since we need to make localized decisions in some cases.
> In lots of places, we can always rely on cacheConf.shouldEvictOnClose() so shouldn't be too burdensome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4589) CacheOnWrite broken in some cases
because it can conflict with evictOnClose
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128015#comment-13128015 ]
Hudson commented on HBASE-4589:
-------------------------------
Integrated in HBase-0.92 #65 (See [https://builds.apache.org/job/HBase-0.92/65/])
HBASE-4589 CacheOnWrite broken in some cases because it can conflict with evictOnClose (jgray)
jgray :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java
> CacheOnWrite broken in some cases because it can conflict with evictOnClose
> ---------------------------------------------------------------------------
>
> Key: HBASE-4589
> URL: https://issues.apache.org/jira/browse/HBASE-4589
> Project: HBase
> Issue Type: Bug
> Components: io
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: HBASE-4589-v1.patch
>
>
> Commit of HBASE-4078 added some extra StoreFile verification which just did an open of a StoreFile reader and then closes it, ensuring there's no exception. If evict-on-close is on, which it is by default, this causes all blocks of a file to be evicted even though it's still open.
> We need to add the boolean into the close call in the way we have booleans for cacheBlocks at some point since we need to make localized decisions in some cases.
> In lots of places, we can always rely on cacheConf.shouldEvictOnClose() so shouldn't be too burdensome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4589) CacheOnWrite broken in some cases
because it can conflict with evictOnClose
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128380#comment-13128380 ]
Hudson commented on HBASE-4589:
-------------------------------
Integrated in HBase-TRUNK #2325 (See [https://builds.apache.org/job/HBase-TRUNK/2325/])
HBASE-4589 CacheOnWrite broken in some cases because it can conflict with evictOnClose (jgray)
jgray :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java
> CacheOnWrite broken in some cases because it can conflict with evictOnClose
> ---------------------------------------------------------------------------
>
> Key: HBASE-4589
> URL: https://issues.apache.org/jira/browse/HBASE-4589
> Project: HBase
> Issue Type: Bug
> Components: io
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: HBASE-4589-v1.patch
>
>
> Commit of HBASE-4078 added some extra StoreFile verification which just did an open of a StoreFile reader and then closes it, ensuring there's no exception. If evict-on-close is on, which it is by default, this causes all blocks of a file to be evicted even though it's still open.
> We need to add the boolean into the close call in the way we have booleans for cacheBlocks at some point since we need to make localized decisions in some cases.
> In lots of places, we can always rely on cacheConf.shouldEvictOnClose() so shouldn't be too burdensome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4589) CacheOnWrite broken in some cases
because it can conflict with evictOnClose
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127317#comment-13127317 ]
stack commented on HBASE-4589:
------------------------------
Ignore my earlier mail. You are on this already.
+1 on patch. Adding flag to the close is ugly but you keep the default of no arg so that should work.
> CacheOnWrite broken in some cases because it can conflict with evictOnClose
> ---------------------------------------------------------------------------
>
> Key: HBASE-4589
> URL: https://issues.apache.org/jira/browse/HBASE-4589
> Project: HBase
> Issue Type: Bug
> Components: io
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: HBASE-4589-v1.patch
>
>
> Commit of HBASE-4078 added some extra StoreFile verification which just did an open of a StoreFile reader and then closes it, ensuring there's no exception. If evict-on-close is on, which it is by default, this causes all blocks of a file to be evicted even though it's still open.
> We need to add the boolean into the close call in the way we have booleans for cacheBlocks at some point since we need to make localized decisions in some cases.
> In lots of places, we can always rely on cacheConf.shouldEvictOnClose() so shouldn't be too burdensome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4589) CacheOnWrite broken in some cases
because it can conflict with evictOnClose
Posted by "Jonathan Gray (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-4589:
---------------------------------
Status: Patch Available (was: Open)
> CacheOnWrite broken in some cases because it can conflict with evictOnClose
> ---------------------------------------------------------------------------
>
> Key: HBASE-4589
> URL: https://issues.apache.org/jira/browse/HBASE-4589
> Project: HBase
> Issue Type: Bug
> Components: io
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: HBASE-4589-v1.patch
>
>
> Commit of HBASE-4078 added some extra StoreFile verification which just did an open of a StoreFile reader and then closes it, ensuring there's no exception. If evict-on-close is on, which it is by default, this causes all blocks of a file to be evicted even though it's still open.
> We need to add the boolean into the close call in the way we have booleans for cacheBlocks at some point since we need to make localized decisions in some cases.
> In lots of places, we can always rely on cacheConf.shouldEvictOnClose() so shouldn't be too burdensome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4589) CacheOnWrite broken in some cases
because it can conflict with evictOnClose
Posted by "Jonathan Gray (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-4589:
---------------------------------
Attachment: HBASE-4589-v1.patch
As advertised. Tests passing.
> CacheOnWrite broken in some cases because it can conflict with evictOnClose
> ---------------------------------------------------------------------------
>
> Key: HBASE-4589
> URL: https://issues.apache.org/jira/browse/HBASE-4589
> Project: HBase
> Issue Type: Bug
> Components: io
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: HBASE-4589-v1.patch
>
>
> Commit of HBASE-4078 added some extra StoreFile verification which just did an open of a StoreFile reader and then closes it, ensuring there's no exception. If evict-on-close is on, which it is by default, this causes all blocks of a file to be evicted even though it's still open.
> We need to add the boolean into the close call in the way we have booleans for cacheBlocks at some point since we need to make localized decisions in some cases.
> In lots of places, we can always rely on cacheConf.shouldEvictOnClose() so shouldn't be too burdensome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira