You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2010/06/18 04:03:26 UTC
[jira] Created: (HBASE-2743) Script to drop N regions from a table
and then patch hole the hole by inserting a new hole spanning region to
meta.
Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
-------------------------------------------------------------------------------------------------------------------
Key: HBASE-2743
URL: https://issues.apache.org/jira/browse/HBASE-2743
Project: HBase
Issue Type: Task
Reporter: stack
Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2743) Script to drop N regions from a
table and then patch hole the hole by inserting a new hole spanning region
to meta.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880290#action_12880290 ]
stack commented on HBASE-2743:
------------------------------
I put the scripts here instead: http://github.com/saintstack/hbase_bin_scripts Latest versions have better documentation on their heads.
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2743) Script to drop N regions from a table
and then patch hole the hole by inserting a new hole spanning region to
meta.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2743:
-------------------------
Attachment: (was: excise_regions.rb)
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: excise_regions.rb, plug_hole.rb
>
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2743) Script to drop N regions from a table
and then patch hole the hole by inserting a new hole spanning region to
meta.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2743:
-------------------------
Attachment: excise_regions.rb
plug_hole.rb
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: excise_regions.rb, excise_regions.rb, plug_hole.rb
>
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2743) Script to drop N regions from a
table and then patch hole the hole by inserting a new hole spanning region
to meta.
Posted by "Daniel Einspanjer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882949#action_12882949 ]
Daniel Einspanjer commented on HBASE-2743:
------------------------------------------
Yes. If we've generated 6k regions since 2010-06-10, that's around 200 per day. That means it would take us approximately 200 days of the same traffic shape to hit 40000 again. We might want to try to tune that number a little more, but it doesn't seem terrible where it is.
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2743) Script to drop N regions from a table
and then patch hole the hole by inserting a new hole spanning region to
meta.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2743:
-------------------------
Attachment: (was: excise_regions.rb)
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: excise_regions.rb, plug_hole.rb
>
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2743) Script to drop N regions from a table
and then patch hole the hole by inserting a new hole spanning region to
meta.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2743:
-------------------------
Attachment: (was: plug_hole.rb)
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2743) Script to drop N regions from a table
and then patch hole the hole by inserting a new hole spanning region to
meta.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2743:
-------------------------
Attachment: excise_regions.rb
plug_hole.rb
Testing, the problem is better addressed with two scripts... one to do the offlining, close and delete with another to plug the hole.
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: excise_regions.rb, excise_regions.rb, plug_hole.rb
>
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2743) Script to drop N regions from a
table and then patch hole the hole by inserting a new hole spanning region
to meta.
Posted by "Daniel Einspanjer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882898#action_12882898 ]
Daniel Einspanjer commented on HBASE-2743:
------------------------------------------
I just used this script today to relieve burden on our cluster from too many regions. Details of the operation with some before/after charts are here:
https://bugzilla.mozilla.org/show_bug.cgi?id=574998
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2743) Script to drop N regions from a table
and then patch hole the hole by inserting a new hole spanning region to
meta.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2743:
-------------------------
Attachment: (was: excise_regions.rb)
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: plug_hole.rb
>
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2743) Script to drop N regions from a
table and then patch hole the hole by inserting a new hole spanning region
to meta.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882905#action_12882905 ]
stack commented on HBASE-2743:
------------------------------
@Daniel It looks it worked then? 41k to 6k regions?
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2743) Script to drop N regions from a table
and then patch hole the hole by inserting a new hole spanning region to
meta.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2743:
-------------------------
Attachment: excise_regions.rb
Script that offlines regions in the first run and then on subsequent run, closes and deletes regions finishing by adding up a hole-plugging region to meta. Needs more testing.
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: excise_regions.rb
>
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2743) Script to drop N regions from a
table and then patch hole the hole by inserting a new hole spanning region
to meta.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880256#action_12880256 ]
stack commented on HBASE-2743:
------------------------------
This version records changes made into designated table so we can undo at later date.
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: excise_regions.rb, plug_hole.rb
>
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2743) Script to drop N regions from a table
and then patch hole the hole by inserting a new hole spanning region to
meta.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2743:
-------------------------
Attachment: (was: plug_hole.rb)
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: excise_regions.rb, excise_regions.rb, plug_hole.rb
>
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2743) Script to drop N regions from a
table and then patch hole the hole by inserting a new hole spanning region
to meta.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880288#action_12880288 ]
stack commented on HBASE-2743:
------------------------------
Here is how the scripts work:
Run excise_regions.rb as follows:
{code}
./bin/hbase org.jruby.Main bin/excise_regions.rb TestTable 0000107136 0104707253 archive
{code}
The keys provided must be actual region startkeys that are present in the table. 'archive' is the name of the table we archive what we've done into. It must be present for this script to work. It must have a column family named 'info'. Do:
hbase> create 'archive', 'info'
... to create the table.
Run the above multiple times till no more 'Offlined=' and 'Closed and delete=' messages.
You may get an NPE from time to time. Thats OK. The master is asked to run parts of this job and when it goes to execute, the info it needs may not be present in .META. (Master needs a patch to not NPE).
When no more messages, then you all regions between the passed key range will have been removed from .META. (and closed out on the regionservers).
The regions that were removed from .META. will be listed in the passed archive table ('archive' in the cmdline above).
You will now have a hole in your table.
To plug the hole, run the following:
{code}
./bin/hbase org.jruby.Main bin/plug_hole.rb TestTable 0000107136 0104707253
{code}
This adds a region that spans the passed keys.
> Script to drop N regions from a table and then patch hole the hole by inserting a new hole spanning region to meta.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-2743
> URL: https://issues.apache.org/jira/browse/HBASE-2743
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: excise_regions.rb, plug_hole.rb
>
>
> Script to help out our mozilla buddies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.