You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Jim Kellerman (JIRA)" <ji...@apache.org> on 2008/10/22 00:44:44 UTC
[jira] Created: (HBASE-947) Major compaction should remove deletes
as well as the deleted cell
Major compaction should remove deletes as well as the deleted cell
------------------------------------------------------------------
Key: HBASE-947
URL: https://issues.apache.org/jira/browse/HBASE-947
Project: Hadoop HBase
Issue Type: Improvement
Reporter: Jim Kellerman
Fix For: 0.19.0
Currently major compactions retains both deletes and the deleted cell. It should remove both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-947) [Optimization] Major compaction should
remove deletes as well as the deleted cell
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-947:
------------------------
Priority: Minor (was: Major)
Summary: [Optimization] Major compaction should remove deletes as well as the deleted cell (was: Major compaction should remove deletes as well as the deleted cell)
To be clear, Jim is suggesting an optimization, a minor one I believe.
Currently cells are only let go from a store file on a major compaction for following reasons:
+ if > MAX_VERSIONS cells or
+ if cell timestamp is older than the configured TTL.
Under this regimen, we could keep around deletes though the cell they overshadow may no longer be present (probably because > MAX_VERSIONS). The suggestion here is that in this one case, we'd let go of the delete cell too (if no corresponding deleted cell).
We don't want to let go of the deleted cell just because there is a delete record on major compaction because then a user who took out a scanner that was behind the delete cells timestamp but in front of the deleted cells timestamp would get different results dependent on whether a major compaction had run or not. Current rule, till we decide otherwise, is that they'd only see a different result if MAX_VERSIONS or TTL had been exceeded.
Marking this issue minor rather than major.
> [Optimization] Major compaction should remove deletes as well as the deleted cell
> ---------------------------------------------------------------------------------
>
> Key: HBASE-947
> URL: https://issues.apache.org/jira/browse/HBASE-947
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jim Kellerman
> Priority: Minor
> Fix For: 0.19.0
>
>
> Currently major compactions retains both deletes and the deleted cell. It should remove both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-947) [Optimization] Major compaction should
remove deletes as well as the deleted cell
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-947.
-------------------------
Resolution: Fixed
Fix Version/s: (was: 0.20.0)
0.19.0
Hadoop Flags: [Incompatible change]
Committed. Now, on major compactions, deletes and what they overshadow are NOT added to the majorly-compacted file.
> [Optimization] Major compaction should remove deletes as well as the deleted cell
> ---------------------------------------------------------------------------------
>
> Key: HBASE-947
> URL: https://issues.apache.org/jira/browse/HBASE-947
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jim Kellerman
> Assignee: stack
> Priority: Minor
> Fix For: 0.19.0
>
>
> Currently major compactions retains both deletes and the deleted cell. It should remove both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-947) [Optimization] Major compaction should
remove deletes as well as the deleted cell
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-947:
------------------------
Fix Version/s: (was: 0.19.0)
0.20.0
Chatting w/ Jon Gray, he actually needs HBASE-1001. Moving this out of 0.19.0.
> [Optimization] Major compaction should remove deletes as well as the deleted cell
> ---------------------------------------------------------------------------------
>
> Key: HBASE-947
> URL: https://issues.apache.org/jira/browse/HBASE-947
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jim Kellerman
> Priority: Minor
> Fix For: 0.20.0
>
>
> Currently major compactions retains both deletes and the deleted cell. It should remove both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-947) [Optimization] Major compaction should
remove deletes as well as the deleted cell
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-947:
------------------------
Fix Version/s: (was: 0.19.0)
Moving out of 0.19.0. Its a bit of work and its a minor optimization.
> [Optimization] Major compaction should remove deletes as well as the deleted cell
> ---------------------------------------------------------------------------------
>
> Key: HBASE-947
> URL: https://issues.apache.org/jira/browse/HBASE-947
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jim Kellerman
> Priority: Minor
>
> Currently major compactions retains both deletes and the deleted cell. It should remove both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-947) [Optimization] Major compaction should
remove deletes as well as the deleted cell
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-947:
------------------------
Fix Version/s: 0.19.0
Moving back into 0.19.0. Jon Gray started crying when I moved it out.
> [Optimization] Major compaction should remove deletes as well as the deleted cell
> ---------------------------------------------------------------------------------
>
> Key: HBASE-947
> URL: https://issues.apache.org/jira/browse/HBASE-947
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jim Kellerman
> Priority: Minor
> Fix For: 0.19.0
>
>
> Currently major compactions retains both deletes and the deleted cell. It should remove both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-947) [Optimization] Major compaction
should remove deletes as well as the deleted cell
Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12646145#action_12646145 ]
Billy Pearson commented on HBASE-947:
-------------------------------------
Also look at this point if we have row&column has data and then deleted and never add a new entry to that same row
column the data says around forever if ttl is not in uses. this cause us to use more data space and memory in the map file index.
> [Optimization] Major compaction should remove deletes as well as the deleted cell
> ---------------------------------------------------------------------------------
>
> Key: HBASE-947
> URL: https://issues.apache.org/jira/browse/HBASE-947
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jim Kellerman
> Priority: Minor
> Fix For: 0.19.0
>
>
> Currently major compactions retains both deletes and the deleted cell. It should remove both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-947) [Optimization] Major compaction should
remove deletes as well as the deleted cell
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack reassigned HBASE-947:
---------------------------
Assignee: stack
> [Optimization] Major compaction should remove deletes as well as the deleted cell
> ---------------------------------------------------------------------------------
>
> Key: HBASE-947
> URL: https://issues.apache.org/jira/browse/HBASE-947
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jim Kellerman
> Assignee: stack
> Priority: Minor
> Fix For: 0.20.0
>
>
> Currently major compactions retains both deletes and the deleted cell. It should remove both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.