You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2008/08/07 05:51:44 UTC

[jira] Created: (HBASE-800) support HTD and HCD get/set attribute in shell, Thrift, and REST interfaces

support HTD and HCD get/set attribute in shell, Thrift, and REST interfaces
---------------------------------------------------------------------------

                 Key: HBASE-800
                 URL: https://issues.apache.org/jira/browse/HBASE-800
             Project: Hadoop HBase
          Issue Type: Improvement
          Components: client, rest, thrift
    Affects Versions: 0.2.1
            Reporter: Andrew Purtell
            Assignee: Andrew Purtell
            Priority: Minor


>From Billy Pearson on hbase-users@

Hey Andrew

Do we have plans to include setMaxFileSize for the shell,thrift,rest?
So non java users can change this as needed with out having to learn java.

Billy

"Andrew Purtell" <ap...@yahoo.com> wrote in message news:189371.9860.qm@web65516.mail.ac4.yahoo.com...
> Hello David,
>
> Current trunk (upcoming 0.2.0) has support for per-table metadata. See
> https://issues.apache.org/jira/browse/HBASE-42 and
> https://issues.apache.org/jira/browse/HBASE-62.
>
> So maybe you can set the split threshold quite low for the table in
> question?
>
> The default is 256MB (268435456), set globally for all tables in the HBase
> configuration as "hbase.hregion.max.filesize". However it's reasonable to
> set it as low as the DFS blocksize. The guidance for a typical HBase
> installation is to set the DFS blocksize to 8MB (8388608), instead of the
> default 64MB.
>
> At create time:
>
>  HTableDescriptor htd = new HTableDescriptor("foo");
>  htd.setMaxFileSize(8388608);
>  ...
>  HBaseAdmin admin = new HBaseAdmin(hconf);
>  admin.createTable(htd);
>
> If the table already exists:
>
>  HTable table = new HTable(hconf, "foo");
>  admin.disableTable("foo");
>  // make a read-write descriptor
>  HTableDescriptor htd =
>    new HTableDescriptor(table.getTableDescriptor());
>  htd.setMaxFileSize(83388608);
>  admin.modifyTableMeta("foo", htd);
>  admin.enableTable("foo");
>
> Hope this helps,
>
>   - Andy
>
>> From: David Alves
>> <dr...@criticalsoftware.com>
>> Subject: Region Splits
>> To: "hbase-user@hadoop.apache.org"
>> <hb...@hadoop.apache.org>
>> Date: Thursday, July 31, 2008, 6:06 AM
> [...]
>> I use hbase (amongst other things) to crawl some repos of infomation
>> and util now I've been using the Nutch segment generation paradigm.
>> I would very much like to skip the segment generation step using
>> hbase as source and sink directly but in order to do that I would
>> need to either allow more that one split to be generated for a
>> single region or make the regions in this particular table split
>> with much less entries than other tables.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Assigned: (HBASE-800) support HTD and HCD get/set attribute in shell, Thrift, and REST interfaces

Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Purtell reassigned HBASE-800:
------------------------------------

    Assignee:     (was: Andrew Purtell)

> support HTD and HCD get/set attribute in shell, Thrift, and REST interfaces
> ---------------------------------------------------------------------------
>
>                 Key: HBASE-800
>                 URL: https://issues.apache.org/jira/browse/HBASE-800
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: client, rest, thrift
>    Affects Versions: 0.2.1
>            Reporter: Andrew Purtell
>            Priority: Minor
>
> From Billy Pearson on hbase-users@
> Hey Andrew
> Do we have plans to include setMaxFileSize for the shell,thrift,rest?
> So non java users can change this as needed with out having to learn java.
> Billy
> "Andrew Purtell" <ap...@yahoo.com> wrote in message news:189371.9860.qm@web65516.mail.ac4.yahoo.com...
> > Hello David,
> >
> > Current trunk (upcoming 0.2.0) has support for per-table metadata. See
> > https://issues.apache.org/jira/browse/HBASE-42 and
> > https://issues.apache.org/jira/browse/HBASE-62.
> >
> > So maybe you can set the split threshold quite low for the table in
> > question?
> >
> > The default is 256MB (268435456), set globally for all tables in the HBase
> > configuration as "hbase.hregion.max.filesize". However it's reasonable to
> > set it as low as the DFS blocksize. The guidance for a typical HBase
> > installation is to set the DFS blocksize to 8MB (8388608), instead of the
> > default 64MB.
> >
> > At create time:
> >
> >  HTableDescriptor htd = new HTableDescriptor("foo");
> >  htd.setMaxFileSize(8388608);
> >  ...
> >  HBaseAdmin admin = new HBaseAdmin(hconf);
> >  admin.createTable(htd);
> >
> > If the table already exists:
> >
> >  HTable table = new HTable(hconf, "foo");
> >  admin.disableTable("foo");
> >  // make a read-write descriptor
> >  HTableDescriptor htd =
> >    new HTableDescriptor(table.getTableDescriptor());
> >  htd.setMaxFileSize(83388608);
> >  admin.modifyTableMeta("foo", htd);
> >  admin.enableTable("foo");
> >
> > Hope this helps,
> >
> >   - Andy
> >
> >> From: David Alves
> >> <dr...@criticalsoftware.com>
> >> Subject: Region Splits
> >> To: "hbase-user@hadoop.apache.org"
> >> <hb...@hadoop.apache.org>
> >> Date: Thursday, July 31, 2008, 6:06 AM
> > [...]
> >> I use hbase (amongst other things) to crawl some repos of infomation
> >> and util now I've been using the Nutch segment generation paradigm.
> >> I would very much like to skip the segment generation step using
> >> hbase as source and sink directly but in order to do that I would
> >> need to either allow more that one split to be generated for a
> >> single region or make the regions in this particular table split
> >> with much less entries than other tables.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-800) support HTD and HCD get/set attribute in shell, Thrift, and REST interfaces

Posted by "stack (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12623166#action_12623166 ] 

stack commented on HBASE-800:
-----------------------------

Also, the new RowLock stuff should be available via thrift.


> support HTD and HCD get/set attribute in shell, Thrift, and REST interfaces
> ---------------------------------------------------------------------------
>
>                 Key: HBASE-800
>                 URL: https://issues.apache.org/jira/browse/HBASE-800
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: client, rest, thrift
>    Affects Versions: 0.2.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>
> From Billy Pearson on hbase-users@
> Hey Andrew
> Do we have plans to include setMaxFileSize for the shell,thrift,rest?
> So non java users can change this as needed with out having to learn java.
> Billy
> "Andrew Purtell" <ap...@yahoo.com> wrote in message news:189371.9860.qm@web65516.mail.ac4.yahoo.com...
> > Hello David,
> >
> > Current trunk (upcoming 0.2.0) has support for per-table metadata. See
> > https://issues.apache.org/jira/browse/HBASE-42 and
> > https://issues.apache.org/jira/browse/HBASE-62.
> >
> > So maybe you can set the split threshold quite low for the table in
> > question?
> >
> > The default is 256MB (268435456), set globally for all tables in the HBase
> > configuration as "hbase.hregion.max.filesize". However it's reasonable to
> > set it as low as the DFS blocksize. The guidance for a typical HBase
> > installation is to set the DFS blocksize to 8MB (8388608), instead of the
> > default 64MB.
> >
> > At create time:
> >
> >  HTableDescriptor htd = new HTableDescriptor("foo");
> >  htd.setMaxFileSize(8388608);
> >  ...
> >  HBaseAdmin admin = new HBaseAdmin(hconf);
> >  admin.createTable(htd);
> >
> > If the table already exists:
> >
> >  HTable table = new HTable(hconf, "foo");
> >  admin.disableTable("foo");
> >  // make a read-write descriptor
> >  HTableDescriptor htd =
> >    new HTableDescriptor(table.getTableDescriptor());
> >  htd.setMaxFileSize(83388608);
> >  admin.modifyTableMeta("foo", htd);
> >  admin.enableTable("foo");
> >
> > Hope this helps,
> >
> >   - Andy
> >
> >> From: David Alves
> >> <dr...@criticalsoftware.com>
> >> Subject: Region Splits
> >> To: "hbase-user@hadoop.apache.org"
> >> <hb...@hadoop.apache.org>
> >> Date: Thursday, July 31, 2008, 6:06 AM
> > [...]
> >> I use hbase (amongst other things) to crawl some repos of infomation
> >> and util now I've been using the Nutch segment generation paradigm.
> >> I would very much like to skip the segment generation step using
> >> hbase as source and sink directly but in order to do that I would
> >> need to either allow more that one split to be generated for a
> >> single region or make the regions in this particular table split
> >> with much less entries than other tables.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-800) support HTD and HCD get/set attribute in shell, Thrift, and REST interfaces

Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639690#action_12639690 ] 

Billy Pearson commented on HBASE-800:
-------------------------------------

Might change this task in to some sub-task one for rest,shell,thrift for HTD and HCD so they can be handled one by one


> support HTD and HCD get/set attribute in shell, Thrift, and REST interfaces
> ---------------------------------------------------------------------------
>
>                 Key: HBASE-800
>                 URL: https://issues.apache.org/jira/browse/HBASE-800
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: client, rest, thrift
>    Affects Versions: 0.2.1
>            Reporter: Andrew Purtell
>            Priority: Minor
>
> From Billy Pearson on hbase-users@
> Hey Andrew
> Do we have plans to include setMaxFileSize for the shell,thrift,rest?
> So non java users can change this as needed with out having to learn java.
> Billy
> "Andrew Purtell" <ap...@yahoo.com> wrote in message news:189371.9860.qm@web65516.mail.ac4.yahoo.com...
> > Hello David,
> >
> > Current trunk (upcoming 0.2.0) has support for per-table metadata. See
> > https://issues.apache.org/jira/browse/HBASE-42 and
> > https://issues.apache.org/jira/browse/HBASE-62.
> >
> > So maybe you can set the split threshold quite low for the table in
> > question?
> >
> > The default is 256MB (268435456), set globally for all tables in the HBase
> > configuration as "hbase.hregion.max.filesize". However it's reasonable to
> > set it as low as the DFS blocksize. The guidance for a typical HBase
> > installation is to set the DFS blocksize to 8MB (8388608), instead of the
> > default 64MB.
> >
> > At create time:
> >
> >  HTableDescriptor htd = new HTableDescriptor("foo");
> >  htd.setMaxFileSize(8388608);
> >  ...
> >  HBaseAdmin admin = new HBaseAdmin(hconf);
> >  admin.createTable(htd);
> >
> > If the table already exists:
> >
> >  HTable table = new HTable(hconf, "foo");
> >  admin.disableTable("foo");
> >  // make a read-write descriptor
> >  HTableDescriptor htd =
> >    new HTableDescriptor(table.getTableDescriptor());
> >  htd.setMaxFileSize(83388608);
> >  admin.modifyTableMeta("foo", htd);
> >  admin.enableTable("foo");
> >
> > Hope this helps,
> >
> >   - Andy
> >
> >> From: David Alves
> >> <dr...@criticalsoftware.com>
> >> Subject: Region Splits
> >> To: "hbase-user@hadoop.apache.org"
> >> <hb...@hadoop.apache.org>
> >> Date: Thursday, July 31, 2008, 6:06 AM
> > [...]
> >> I use hbase (amongst other things) to crawl some repos of infomation
> >> and util now I've been using the Nutch segment generation paradigm.
> >> I would very much like to skip the segment generation step using
> >> hbase as source and sink directly but in order to do that I would
> >> need to either allow more that one split to be generated for a
> >> single region or make the regions in this particular table split
> >> with much less entries than other tables.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-800) support HTD and HCD get/set attribute in shell, Thrift, and REST interfaces

Posted by "Lars Francke (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12785107#action_12785107 ] 

Lars Francke commented on HBASE-800:
------------------------------------

Thrift will have modifyTable, getTableDescriptor and createTable with a TableDescriptor with the new Thrift API developed in HBASE-1744. This would solve a part of this ticket.

> support HTD and HCD get/set attribute in shell, Thrift, and REST interfaces
> ---------------------------------------------------------------------------
>
>                 Key: HBASE-800
>                 URL: https://issues.apache.org/jira/browse/HBASE-800
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: client, rest, thrift
>    Affects Versions: 0.2.1
>            Reporter: Andrew Purtell
>            Priority: Minor
>
> From Billy Pearson on hbase-users@
> Hey Andrew
> Do we have plans to include setMaxFileSize for the shell,thrift,rest?
> So non java users can change this as needed with out having to learn java.
> Billy
> "Andrew Purtell" <ap...@yahoo.com> wrote in message news:189371.9860.qm@web65516.mail.ac4.yahoo.com...
> > Hello David,
> >
> > Current trunk (upcoming 0.2.0) has support for per-table metadata. See
> > https://issues.apache.org/jira/browse/HBASE-42 and
> > https://issues.apache.org/jira/browse/HBASE-62.
> >
> > So maybe you can set the split threshold quite low for the table in
> > question?
> >
> > The default is 256MB (268435456), set globally for all tables in the HBase
> > configuration as "hbase.hregion.max.filesize". However it's reasonable to
> > set it as low as the DFS blocksize. The guidance for a typical HBase
> > installation is to set the DFS blocksize to 8MB (8388608), instead of the
> > default 64MB.
> >
> > At create time:
> >
> >  HTableDescriptor htd = new HTableDescriptor("foo");
> >  htd.setMaxFileSize(8388608);
> >  ...
> >  HBaseAdmin admin = new HBaseAdmin(hconf);
> >  admin.createTable(htd);
> >
> > If the table already exists:
> >
> >  HTable table = new HTable(hconf, "foo");
> >  admin.disableTable("foo");
> >  // make a read-write descriptor
> >  HTableDescriptor htd =
> >    new HTableDescriptor(table.getTableDescriptor());
> >  htd.setMaxFileSize(83388608);
> >  admin.modifyTableMeta("foo", htd);
> >  admin.enableTable("foo");
> >
> > Hope this helps,
> >
> >   - Andy
> >
> >> From: David Alves
> >> <dr...@criticalsoftware.com>
> >> Subject: Region Splits
> >> To: "hbase-user@hadoop.apache.org"
> >> <hb...@hadoop.apache.org>
> >> Date: Thursday, July 31, 2008, 6:06 AM
> > [...]
> >> I use hbase (amongst other things) to crawl some repos of infomation
> >> and util now I've been using the Nutch segment generation paradigm.
> >> I would very much like to skip the segment generation step using
> >> hbase as source and sink directly but in order to do that I would
> >> need to either allow more that one split to be generated for a
> >> single region or make the regions in this particular table split
> >> with much less entries than other tables.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.