You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "chunhui shen (JIRA)" <ji...@apache.org> on 2012/06/13 05:03:42 UTC

[jira] [Created] (HBASE-6204) Improvement for opening region on the regionserver

chunhui shen created HBASE-6204:
-----------------------------------

             Summary: Improvement for opening region on the regionserver
                 Key: HBASE-6204
                 URL: https://issues.apache.org/jira/browse/HBASE-6204
             Project: HBase
          Issue Type: Improvement
          Components: regionserver
            Reporter: chunhui shen
            Assignee: chunhui shen


If one table has much regions, like 100k regions.
We would find regionserver open region very slowly.
Opening region in parallel on the rs will be closed to serially.

The following code is the detail:
{code}
public RegionOpeningState openRegion(HRegionInfo region, int versionOfOfflineNode){
...
HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
...

public HTableDescriptor get(final String tablename){
...
long modtime = getTableInfoModtime(this.fs, this.rootdir, tablename);
...
}
}
{code}

getTableInfoModtime->getTableInfoPath->getTableInfoPath->FSUtils.listStatus()

if one table has much regions, FSUtils.listStatus() will take much time

How to improve the above code?
I think an easy way is that make a dir (called .tableinfos) in the table dir and move the files .tableinfo.* to that dir.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6204) Improvement for opening region on the regionserver

Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294143#comment-13294143 ] 

chunhui shen commented on HBASE-6204:
-------------------------------------

@ram
Thanks, HBASE-5998 is good.
So close this issue.
                
> Improvement for opening region on the regionserver
> --------------------------------------------------
>
>                 Key: HBASE-6204
>                 URL: https://issues.apache.org/jira/browse/HBASE-6204
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>
> If one table has many regions, like 100k regions.
> We would find regionserver open region very slowly.
> Opening region in parallel on the rs will be closed to serially.
> The following code is the detail:
> {code}
> public RegionOpeningState openRegion(HRegionInfo region, int versionOfOfflineNode){
> ...
> HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
> ...
> public HTableDescriptor get(final String tablename){
> ...
> long modtime = getTableInfoModtime(this.fs, this.rootdir, tablename);
> ...
> }
> }
> {code}
> getTableInfoModtime->getTableInfoPath->getTableInfoPath->FSUtils.listStatus()
> if one table has many regions, FSUtils.listStatus() will take much time
> How to improve the above code?
> I think an easy way is that make a dir (called .tableinfos) in the table dir and move the files .tableinfo.* to that dir.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6204) Improvement for opening region on the regionserver

Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-6204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

chunhui shen updated HBASE-6204:
--------------------------------

    Description: 
If one table has many regions, like 100k regions.
We would find regionserver open region very slowly.
Opening region in parallel on the rs will be closed to serially.

The following code is the detail:
{code}
public RegionOpeningState openRegion(HRegionInfo region, int versionOfOfflineNode){
...
HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
...

public HTableDescriptor get(final String tablename){
...
long modtime = getTableInfoModtime(this.fs, this.rootdir, tablename);
...
}
}
{code}

getTableInfoModtime->getTableInfoPath->getTableInfoPath->FSUtils.listStatus()

if one table has much regions, FSUtils.listStatus() will take much time

How to improve the above code?
I think an easy way is that make a dir (called .tableinfos) in the table dir and move the files .tableinfo.* to that dir.

  was:
If one table has much regions, like 100k regions.
We would find regionserver open region very slowly.
Opening region in parallel on the rs will be closed to serially.

The following code is the detail:
{code}
public RegionOpeningState openRegion(HRegionInfo region, int versionOfOfflineNode){
...
HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
...

public HTableDescriptor get(final String tablename){
...
long modtime = getTableInfoModtime(this.fs, this.rootdir, tablename);
...
}
}
{code}

getTableInfoModtime->getTableInfoPath->getTableInfoPath->FSUtils.listStatus()

if one table has much regions, FSUtils.listStatus() will take much time

How to improve the above code?
I think an easy way is that make a dir (called .tableinfos) in the table dir and move the files .tableinfo.* to that dir.

    
> Improvement for opening region on the regionserver
> --------------------------------------------------
>
>                 Key: HBASE-6204
>                 URL: https://issues.apache.org/jira/browse/HBASE-6204
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>
> If one table has many regions, like 100k regions.
> We would find regionserver open region very slowly.
> Opening region in parallel on the rs will be closed to serially.
> The following code is the detail:
> {code}
> public RegionOpeningState openRegion(HRegionInfo region, int versionOfOfflineNode){
> ...
> HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
> ...
> public HTableDescriptor get(final String tablename){
> ...
> long modtime = getTableInfoModtime(this.fs, this.rootdir, tablename);
> ...
> }
> }
> {code}
> getTableInfoModtime->getTableInfoPath->getTableInfoPath->FSUtils.listStatus()
> if one table has much regions, FSUtils.listStatus() will take much time
> How to improve the above code?
> I think an easy way is that make a dir (called .tableinfos) in the table dir and move the files .tableinfo.* to that dir.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6204) Improvement for opening region on the regionserver

Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294133#comment-13294133 ] 

ramkrishna.s.vasudevan commented on HBASE-6204:
-----------------------------------------------

@Chunhui
See HBASE-5998.  May be that's what you are suggesting too. Thanks.
                
> Improvement for opening region on the regionserver
> --------------------------------------------------
>
>                 Key: HBASE-6204
>                 URL: https://issues.apache.org/jira/browse/HBASE-6204
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>
> If one table has many regions, like 100k regions.
> We would find regionserver open region very slowly.
> Opening region in parallel on the rs will be closed to serially.
> The following code is the detail:
> {code}
> public RegionOpeningState openRegion(HRegionInfo region, int versionOfOfflineNode){
> ...
> HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
> ...
> public HTableDescriptor get(final String tablename){
> ...
> long modtime = getTableInfoModtime(this.fs, this.rootdir, tablename);
> ...
> }
> }
> {code}
> getTableInfoModtime->getTableInfoPath->getTableInfoPath->FSUtils.listStatus()
> if one table has many regions, FSUtils.listStatus() will take much time
> How to improve the above code?
> I think an easy way is that make a dir (called .tableinfos) in the table dir and move the files .tableinfo.* to that dir.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6204) Improvement for opening region on the regionserver

Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-6204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294102#comment-13294102 ] 

chunhui shen commented on HBASE-6204:
-------------------------------------

I will make a patc later.
                
> Improvement for opening region on the regionserver
> --------------------------------------------------
>
>                 Key: HBASE-6204
>                 URL: https://issues.apache.org/jira/browse/HBASE-6204
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>
> If one table has many regions, like 100k regions.
> We would find regionserver open region very slowly.
> Opening region in parallel on the rs will be closed to serially.
> The following code is the detail:
> {code}
> public RegionOpeningState openRegion(HRegionInfo region, int versionOfOfflineNode){
> ...
> HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
> ...
> public HTableDescriptor get(final String tablename){
> ...
> long modtime = getTableInfoModtime(this.fs, this.rootdir, tablename);
> ...
> }
> }
> {code}
> getTableInfoModtime->getTableInfoPath->getTableInfoPath->FSUtils.listStatus()
> if one table has many regions, FSUtils.listStatus() will take much time
> How to improve the above code?
> I think an easy way is that make a dir (called .tableinfos) in the table dir and move the files .tableinfo.* to that dir.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6204) Improvement for opening region on the regionserver

Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-6204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

chunhui shen updated HBASE-6204:
--------------------------------

    Description: 
If one table has many regions, like 100k regions.
We would find regionserver open region very slowly.
Opening region in parallel on the rs will be closed to serially.

The following code is the detail:
{code}
public RegionOpeningState openRegion(HRegionInfo region, int versionOfOfflineNode){
...
HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
...

public HTableDescriptor get(final String tablename){
...
long modtime = getTableInfoModtime(this.fs, this.rootdir, tablename);
...
}
}
{code}

getTableInfoModtime->getTableInfoPath->getTableInfoPath->FSUtils.listStatus()

if one table has many regions, FSUtils.listStatus() will take much time

How to improve the above code?
I think an easy way is that make a dir (called .tableinfos) in the table dir and move the files .tableinfo.* to that dir.

  was:
If one table has many regions, like 100k regions.
We would find regionserver open region very slowly.
Opening region in parallel on the rs will be closed to serially.

The following code is the detail:
{code}
public RegionOpeningState openRegion(HRegionInfo region, int versionOfOfflineNode){
...
HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
...

public HTableDescriptor get(final String tablename){
...
long modtime = getTableInfoModtime(this.fs, this.rootdir, tablename);
...
}
}
{code}

getTableInfoModtime->getTableInfoPath->getTableInfoPath->FSUtils.listStatus()

if one table has much regions, FSUtils.listStatus() will take much time

How to improve the above code?
I think an easy way is that make a dir (called .tableinfos) in the table dir and move the files .tableinfo.* to that dir.

    
> Improvement for opening region on the regionserver
> --------------------------------------------------
>
>                 Key: HBASE-6204
>                 URL: https://issues.apache.org/jira/browse/HBASE-6204
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>
> If one table has many regions, like 100k regions.
> We would find regionserver open region very slowly.
> Opening region in parallel on the rs will be closed to serially.
> The following code is the detail:
> {code}
> public RegionOpeningState openRegion(HRegionInfo region, int versionOfOfflineNode){
> ...
> HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
> ...
> public HTableDescriptor get(final String tablename){
> ...
> long modtime = getTableInfoModtime(this.fs, this.rootdir, tablename);
> ...
> }
> }
> {code}
> getTableInfoModtime->getTableInfoPath->getTableInfoPath->FSUtils.listStatus()
> if one table has many regions, FSUtils.listStatus() will take much time
> How to improve the above code?
> I think an easy way is that make a dir (called .tableinfos) in the table dir and move the files .tableinfo.* to that dir.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-6204) Improvement for opening region on the regionserver

Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-6204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

chunhui shen resolved HBASE-6204.
---------------------------------

    Resolution: Duplicate
    
> Improvement for opening region on the regionserver
> --------------------------------------------------
>
>                 Key: HBASE-6204
>                 URL: https://issues.apache.org/jira/browse/HBASE-6204
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>
> If one table has many regions, like 100k regions.
> We would find regionserver open region very slowly.
> Opening region in parallel on the rs will be closed to serially.
> The following code is the detail:
> {code}
> public RegionOpeningState openRegion(HRegionInfo region, int versionOfOfflineNode){
> ...
> HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
> ...
> public HTableDescriptor get(final String tablename){
> ...
> long modtime = getTableInfoModtime(this.fs, this.rootdir, tablename);
> ...
> }
> }
> {code}
> getTableInfoModtime->getTableInfoPath->getTableInfoPath->FSUtils.listStatus()
> if one table has many regions, FSUtils.listStatus() will take much time
> How to improve the above code?
> I think an easy way is that make a dir (called .tableinfos) in the table dir and move the files .tableinfo.* to that dir.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira