You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Adela Maznikar (JIRA)" <ji...@apache.org> on 2012/11/05 23:18:12 UTC
[jira] [Created] (HBASE-7099) create compaction hooks
Adela Maznikar created HBASE-7099:
-------------------------------------
Summary: create compaction hooks
Key: HBASE-7099
URL: https://issues.apache.org/jira/browse/HBASE-7099
Project: HBase
Issue Type: New Feature
Reporter: Adela Maznikar
Assignee: Adela Maznikar
Create compaction hooks which should be used to modify the value of the KV during compaction. The possible modifications would include: skipping the KV, modifying the value of the KV in the desired format, leaving the value as it was.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7099) [89-fb] Create compaction hooks
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491239#comment-13491239 ]
ramkrishna.s.vasudevan commented on HBASE-7099:
-----------------------------------------------
Will this also help us to know what KVs did the compaction scanner skip due to TTL expiry ?
> [89-fb] Create compaction hooks
> --------------------------------
>
> Key: HBASE-7099
> URL: https://issues.apache.org/jira/browse/HBASE-7099
> Project: HBase
> Issue Type: New Feature
> Reporter: Adela Maznikar
> Assignee: Adela Maznikar
>
> Create compaction hooks which should be used to modify the value of the KV during compaction. The possible modifications would include: skipping the KV, modifying the value of the KV in the desired format, leaving the value as it was.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-7099) [89-fb] Create compaction hooks
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell reopened HBASE-7099:
-----------------------------------
> [89-fb] Create compaction hooks
> --------------------------------
>
> Key: HBASE-7099
> URL: https://issues.apache.org/jira/browse/HBASE-7099
> Project: HBase
> Issue Type: New Feature
> Reporter: Adela Maznikar
> Assignee: Adela Maznikar
>
> Create compaction hooks which should be used to modify the value of the KV during compaction. The possible modifications would include: skipping the KV, modifying the value of the KV in the desired format, leaving the value as it was.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7099) [89-fb] Create compaction hooks
Posted by "Kannan Muthukkaruppan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491565#comment-13491565 ]
Kannan Muthukkaruppan commented on HBASE-7099:
----------------------------------------------
We were thinking of this as an "emit/write" time compaction hook. It won't be in the general "read" path, but in compactStores(). By the time a value gets here, TTL purging etc. would have already been done. Whatever survives, and is about to be written to the output (of compaction) file-- this hook will give a chance to intercept & transform/skip the value portion of the KV.
> [89-fb] Create compaction hooks
> --------------------------------
>
> Key: HBASE-7099
> URL: https://issues.apache.org/jira/browse/HBASE-7099
> Project: HBase
> Issue Type: New Feature
> Reporter: Adela Maznikar
> Assignee: Adela Maznikar
>
> Create compaction hooks which should be used to modify the value of the KV during compaction. The possible modifications would include: skipping the KV, modifying the value of the KV in the desired format, leaving the value as it was.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7099) [89-fb] Create compaction hooks
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491246#comment-13491246 ]
Lars Hofhansl commented on HBASE-7099:
--------------------------------------
I come across this use case with surprising frequency: How to track KVs "deleted" during a compaction?
> [89-fb] Create compaction hooks
> --------------------------------
>
> Key: HBASE-7099
> URL: https://issues.apache.org/jira/browse/HBASE-7099
> Project: HBase
> Issue Type: New Feature
> Reporter: Adela Maznikar
> Assignee: Adela Maznikar
>
> Create compaction hooks which should be used to modify the value of the KV during compaction. The possible modifications would include: skipping the KV, modifying the value of the KV in the desired format, leaving the value as it was.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7099) create compaction hooks
Posted by "Adela Maznikar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491042#comment-13491042 ]
Adela Maznikar commented on HBASE-7099:
---------------------------------------
Andrew, I am working on the 89-fb branch. Sorry for the confusion, will modify the summary now. Thanks
> create compaction hooks
> ------------------------
>
> Key: HBASE-7099
> URL: https://issues.apache.org/jira/browse/HBASE-7099
> Project: HBase
> Issue Type: New Feature
> Reporter: Adela Maznikar
> Assignee: Adela Maznikar
>
> Create compaction hooks which should be used to modify the value of the KV during compaction. The possible modifications would include: skipping the KV, modifying the value of the KV in the desired format, leaving the value as it was.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7099) [89-fb] Create compaction hooks
Posted by "Adela Maznikar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adela Maznikar updated HBASE-7099:
----------------------------------
Summary: [89-fb] Create compaction hooks (was: create compaction hooks )
> [89-fb] Create compaction hooks
> --------------------------------
>
> Key: HBASE-7099
> URL: https://issues.apache.org/jira/browse/HBASE-7099
> Project: HBase
> Issue Type: New Feature
> Reporter: Adela Maznikar
> Assignee: Adela Maznikar
>
> Create compaction hooks which should be used to modify the value of the KV during compaction. The possible modifications would include: skipping the KV, modifying the value of the KV in the desired format, leaving the value as it was.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7099) create compaction hooks
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell resolved HBASE-7099.
-----------------------------------
Resolution: Invalid
The existing preCompact hook allows skipping KVs and modifying the values of KVs.
Do this:
{code}
public preCompact(final ObserverContext<RegionCoprocessorEnvironment> c,
final Store store,
final InternalScanner scanner)
{
return new MyCustomInternalScanner(scanner);
}
{code}
Your custom InternalScanner therefore gets to inspect, drop, or change KVs from the wrapped scanner. It is your custom scanner that will be used during compaction.
> create compaction hooks
> ------------------------
>
> Key: HBASE-7099
> URL: https://issues.apache.org/jira/browse/HBASE-7099
> Project: HBase
> Issue Type: New Feature
> Reporter: Adela Maznikar
> Assignee: Adela Maznikar
>
> Create compaction hooks which should be used to modify the value of the KV during compaction. The possible modifications would include: skipping the KV, modifying the value of the KV in the desired format, leaving the value as it was.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7099) [89-fb] Create compaction hooks
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491045#comment-13491045 ]
Andrew Purtell commented on HBASE-7099:
---------------------------------------
My apologies Adela.
> [89-fb] Create compaction hooks
> --------------------------------
>
> Key: HBASE-7099
> URL: https://issues.apache.org/jira/browse/HBASE-7099
> Project: HBase
> Issue Type: New Feature
> Reporter: Adela Maznikar
> Assignee: Adela Maznikar
>
> Create compaction hooks which should be used to modify the value of the KV during compaction. The possible modifications would include: skipping the KV, modifying the value of the KV in the desired format, leaving the value as it was.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira