You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Karthik Ranganathan (Created) (JIRA)" <ji...@apache.org> on 2011/10/25 06:50:32 UTC
[jira] [Created] (HBASE-4661) Ability to export the list of files
for a some or all column families for a given region
Ability to export the list of files for a some or all column families for a given region
----------------------------------------------------------------------------------------
Key: HBASE-4661
URL: https://issues.apache.org/jira/browse/HBASE-4661
Project: HBase
Issue Type: Sub-task
Reporter: Karthik Ranganathan
Assignee: Karthik Ranganathan
Should be able to query the regionservers to figure out the list of files in one/some/all column families for a given regions to determine which files to copy for a backup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4661) Ability to export the list of files
for a some or all column families for a given region
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168281#comment-13168281 ]
Hudson commented on HBASE-4661:
-------------------------------
Integrated in HBase-TRUNK #2541 (See [https://builds.apache.org/job/HBase-TRUNK/2541/])
[jira] [HBASE-4661] get list of store files.
Summary:
1. Ported getStoreFileList API to trunk.
2. Also ported flushRegion API to trunk (with option of flushing if
lastFlushTime < TS).
Test Plan: Tested on 89.
Reviewers: Karthik, Kannan, nspiegelberg, JIRA
Reviewed By: Karthik
CC: HBase Diffs Facebook Group, Karthik, Kannan, stack, madhuvaidya
Differential Revision: 729
karthik :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> Ability to export the list of files for a some or all column families for a given region
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-4661
> URL: https://issues.apache.org/jira/browse/HBASE-4661
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, regionserver
> Reporter: Karthik Ranganathan
> Assignee: Madhuwanti Vaidya
> Fix For: 0.94.0
>
> Attachments: D729.2.patch
>
>
> Should be able to query the regionservers to figure out the list of files in one/some/all column families for a given regions to determine which files to copy for a backup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4661) Ability to export the list of files
for a some or all column families for a given region
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168212#comment-13168212 ]
Hudson commented on HBASE-4661:
-------------------------------
Integrated in HBase-TRUNK-security #30 (See [https://builds.apache.org/job/HBase-TRUNK-security/30/])
[jira] [HBASE-4661] get list of store files.
Summary:
1. Ported getStoreFileList API to trunk.
2. Also ported flushRegion API to trunk (with option of flushing if
lastFlushTime < TS).
Test Plan: Tested on 89.
Reviewers: Karthik, Kannan, nspiegelberg, JIRA
Reviewed By: Karthik
CC: HBase Diffs Facebook Group, Karthik, Kannan, stack, madhuvaidya
Differential Revision: 729
karthik :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> Ability to export the list of files for a some or all column families for a given region
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-4661
> URL: https://issues.apache.org/jira/browse/HBASE-4661
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, regionserver
> Reporter: Karthik Ranganathan
> Assignee: Madhuwanti Vaidya
> Fix For: 0.94.0
>
> Attachments: D729.2.patch
>
>
> Should be able to query the regionservers to figure out the list of files in one/some/all column families for a given regions to determine which files to copy for a backup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4661) Ability to export the list of files
for a some or all column families for a given region
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167857#comment-13167857 ]
Phabricator commented on HBASE-4661:
------------------------------------
stack has commented on the revision "[jira] [HBASE-4661] get list of store files.".
+1
If none of you lads are going to commit it, stick it up on JIRA and I will... good stuff.
REVISION DETAIL
https://reviews.facebook.net/D729
> Ability to export the list of files for a some or all column families for a given region
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-4661
> URL: https://issues.apache.org/jira/browse/HBASE-4661
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, regionserver
> Reporter: Karthik Ranganathan
> Assignee: Madhuwanti Vaidya
> Attachments: D729.2.patch
>
>
> Should be able to query the regionservers to figure out the list of files in one/some/all column families for a given regions to determine which files to copy for a backup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4661) Ability to export the list of files
for a some or all column families for a given region
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167681#comment-13167681 ]
Phabricator commented on HBASE-4661:
------------------------------------
stack has commented on the revision "[jira] [HBASE-4661] get list of store files.".
INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java:88 This interface already has a flushRegion that takes an HRegionInterface. Should we deprecate it? Is this regionName the full name or is it encoded Name?
src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java:118 Is region the right granularity? We'll have to make a bunch of calls. Should we rather allow instead asking the regionserver return all storefiles for a given table (with optional columnfamily qualifier supplied)?
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:2169 Does the flush happen inline or are we just queuing a flush? If former, perhaps return boolean on whether we flushed or no (not important)
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:2189 How do I express region has not been flushed at all? -1?
REVISION DETAIL
https://reviews.facebook.net/D729
> Ability to export the list of files for a some or all column families for a given region
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-4661
> URL: https://issues.apache.org/jira/browse/HBASE-4661
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, regionserver
> Reporter: Karthik Ranganathan
> Assignee: Madhuwanti Vaidya
>
> Should be able to query the regionservers to figure out the list of files in one/some/all column families for a given regions to determine which files to copy for a backup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4661) Ability to export the list of files
for a some or all column families for a given region
Posted by "Karthik Ranganathan (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Ranganathan resolved HBASE-4661.
----------------------------------------
Resolution: Fixed
Fix Version/s: 0.94.0
> Ability to export the list of files for a some or all column families for a given region
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-4661
> URL: https://issues.apache.org/jira/browse/HBASE-4661
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, regionserver
> Reporter: Karthik Ranganathan
> Assignee: Madhuwanti Vaidya
> Fix For: 0.94.0
>
> Attachments: D729.2.patch
>
>
> Should be able to query the regionservers to figure out the list of files in one/some/all column families for a given regions to determine which files to copy for a backup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4661) Ability to export the list of files
for a some or all column families for a given region
Posted by "Karthik Ranganathan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Ranganathan updated HBASE-4661:
---------------------------------------
Assignee: Madhuwanti Vaidya (was: Karthik Ranganathan)
> Ability to export the list of files for a some or all column families for a given region
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-4661
> URL: https://issues.apache.org/jira/browse/HBASE-4661
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, regionserver
> Reporter: Karthik Ranganathan
> Assignee: Madhuwanti Vaidya
>
> Should be able to query the regionservers to figure out the list of files in one/some/all column families for a given regions to determine which files to copy for a backup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4661) Ability to export the list of files
for a some or all column families for a given region
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167859#comment-13167859 ]
Phabricator commented on HBASE-4661:
------------------------------------
Karthik has commented on the revision "[jira] [HBASE-4661] get list of store files.".
I can commit - left it up accepted and uncommitted in case some else wanted to have a look. Now I feel a lot better committing :)
REVISION DETAIL
https://reviews.facebook.net/D729
> Ability to export the list of files for a some or all column families for a given region
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-4661
> URL: https://issues.apache.org/jira/browse/HBASE-4661
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, regionserver
> Reporter: Karthik Ranganathan
> Assignee: Madhuwanti Vaidya
> Attachments: D729.2.patch
>
>
> Should be able to query the regionservers to figure out the list of files in one/some/all column families for a given regions to determine which files to copy for a backup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4661) Ability to export the list of files
for a some or all column families for a given region
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-4661:
-------------------------------
Attachment: D729.2.patch
madhuvaidya updated the revision "[jira] [HBASE-4661] get list of store files.".
Reviewers: Karthik, Kannan, nspiegelberg, JIRA
Added deprecated to old flushRegion.
REVISION DETAIL
https://reviews.facebook.net/D729
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
> Ability to export the list of files for a some or all column families for a given region
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-4661
> URL: https://issues.apache.org/jira/browse/HBASE-4661
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, regionserver
> Reporter: Karthik Ranganathan
> Assignee: Madhuwanti Vaidya
> Attachments: D729.2.patch
>
>
> Should be able to query the regionservers to figure out the list of files in one/some/all column families for a given regions to determine which files to copy for a backup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4661) Ability to export the list of files
for a some or all column families for a given region
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167825#comment-13167825 ]
Phabricator commented on HBASE-4661:
------------------------------------
stack has commented on the revision "[jira] [HBASE-4661] get list of store files.".
+1 Would suggest adding deprecate to old flushRegion on commit.
INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java:88 Yeah, I'd say deprecate the other. Its kinda a pain anyways taking a whole HRI.
src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java:118 Ok. Sounds good.
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:2189 Ok. Sounds good.
REVISION DETAIL
https://reviews.facebook.net/D729
> Ability to export the list of files for a some or all column families for a given region
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-4661
> URL: https://issues.apache.org/jira/browse/HBASE-4661
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, regionserver
> Reporter: Karthik Ranganathan
> Assignee: Madhuwanti Vaidya
>
> Should be able to query the regionservers to figure out the list of files in one/some/all column families for a given regions to determine which files to copy for a backup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4661) Ability to export the list of files
for a some or all column families for a given region
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167818#comment-13167818 ]
Phabricator commented on HBASE-4661:
------------------------------------
madhuvaidya has commented on the revision "[jira] [HBASE-4661] get list of store files.".
INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java:88 In the backup code we used this API (with regionName which is the full name). Not sure - maybe we could deprecate the other API.
src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java:118 We used it at this granularity since we scheduled one map task per region for taking a backup of the table.
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:2169 The flush happens inline. (It's a blocking call)
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:2189 The lastFlushTime is set when the region is initialized which is equivalent to a flush for our purpose. In each map task - we flush a region before asking for the list of store files to backup. We want to protect against the case when a flush goes through, but the map task fails, in which case a flush would be scheduled again if we didn't check lastFlushTime > backupStartTime.
REVISION DETAIL
https://reviews.facebook.net/D729
> Ability to export the list of files for a some or all column families for a given region
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-4661
> URL: https://issues.apache.org/jira/browse/HBASE-4661
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, regionserver
> Reporter: Karthik Ranganathan
> Assignee: Madhuwanti Vaidya
>
> Should be able to query the regionservers to figure out the list of files in one/some/all column families for a given regions to determine which files to copy for a backup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4661) Ability to export the list of files
for a some or all column families for a given region
Posted by "Madhuwanti Vaidya (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166652#comment-13166652 ]
Madhuwanti Vaidya commented on HBASE-4661:
------------------------------------------
https://reviews.facebook.net/D729
> Ability to export the list of files for a some or all column families for a given region
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-4661
> URL: https://issues.apache.org/jira/browse/HBASE-4661
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, regionserver
> Reporter: Karthik Ranganathan
> Assignee: Madhuwanti Vaidya
>
> Should be able to query the regionservers to figure out the list of files in one/some/all column families for a given regions to determine which files to copy for a backup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4661) Ability to export the list of files
for a some or all column families for a given region
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167923#comment-13167923 ]
Phabricator commented on HBASE-4661:
------------------------------------
madhuvaidya has committed the revision "[jira] [HBASE-4661] get list of store files.".
REVISION DETAIL
https://reviews.facebook.net/D729
COMMIT
https://reviews.facebook.net/rHBASE1213487
> Ability to export the list of files for a some or all column families for a given region
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-4661
> URL: https://issues.apache.org/jira/browse/HBASE-4661
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, regionserver
> Reporter: Karthik Ranganathan
> Assignee: Madhuwanti Vaidya
> Fix For: 0.94.0
>
> Attachments: D729.2.patch
>
>
> Should be able to query the regionservers to figure out the list of files in one/some/all column families for a given regions to determine which files to copy for a backup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira