You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "stack (JIRA)" <ji...@apache.org> on 2007/11/01 18:19:51 UTC
[jira] Created: (HADOOP-2138) [hbase] Master should allocate
regions to the regionserver hosting the region data where possible
[hbase] Master should allocate regions to the regionserver hosting the region data where possible
-------------------------------------------------------------------------------------------------
Key: HADOOP-2138
URL: https://issues.apache.org/jira/browse/HADOOP-2138
Project: Hadoop
Issue Type: Improvement
Components: contrib/hbase
Reporter: stack
Priority: Minor
Currently, regions are assigned regionservers based off a basic loading attribute. A factor to include in the assignment calcuation is the location of the region in hdfs; i.e. servers hosting region replicas. If the cluster is such that regionservers are being run on the same nodes as those running hdfs, then ideally the regionserver for a particular region should be running on the same server as hosts a region replica.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2138) [hbase] Master should allocate
regions to the regionserver hosting the region data where possible
Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12554873 ]
Billy Pearson commented on HADOOP-2138:
---------------------------------------
>From what I understand about writing files to data nodes I do not thank is is a major problem. If the regions are not located on the host serving server on the first compaction they would be stored local first and stay local from there on out unless the current servers hard drive is full and it has to store it on a different server. Over all the region servers would self fix this problem by default over time.
So only time this would be an issue is on a restart or after a failed region server.
> [hbase] Master should allocate regions to the regionserver hosting the region data where possible
> -------------------------------------------------------------------------------------------------
>
> Key: HADOOP-2138
> URL: https://issues.apache.org/jira/browse/HADOOP-2138
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
>
> Currently, regions are assigned regionservers based off a basic loading attribute. A factor to include in the assignment calcuation is the location of the region in hdfs; i.e. servers hosting region replicas. If the cluster is such that regionservers are being run on the same nodes as those running hdfs, then ideally the regionserver for a particular region should be running on the same server as hosts a region replica.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2138) [hbase] Master should allocate
regions to the regionserver hosting the region data where possible
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12549737 ]
stack commented on HADOOP-2138:
-------------------------------
This might actually already be happening. Any time I've seen a regionserver reading from hdfs, it seems to be getting its data from the local datanode. Verify.
> [hbase] Master should allocate regions to the regionserver hosting the region data where possible
> -------------------------------------------------------------------------------------------------
>
> Key: HADOOP-2138
> URL: https://issues.apache.org/jira/browse/HADOOP-2138
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
>
> Currently, regions are assigned regionservers based off a basic loading attribute. A factor to include in the assignment calcuation is the location of the region in hdfs; i.e. servers hosting region replicas. If the cluster is such that regionservers are being run on the same nodes as those running hdfs, then ideally the regionserver for a particular region should be running on the same server as hosts a region replica.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2138) [hbase] Master should allocate
regions to the regionserver hosting the region data where possible
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury updated HADOOP-2138:
----------------------------------
Priority: Major (was: Minor)
This seems like it should be an important issue, since it should significantly improve the performance of the cluster (faster, less network traffic). Elevating to Major.
> [hbase] Master should allocate regions to the regionserver hosting the region data where possible
> -------------------------------------------------------------------------------------------------
>
> Key: HADOOP-2138
> URL: https://issues.apache.org/jira/browse/HADOOP-2138
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
>
> Currently, regions are assigned regionservers based off a basic loading attribute. A factor to include in the assignment calcuation is the location of the region in hdfs; i.e. servers hosting region replicas. If the cluster is such that regionservers are being run on the same nodes as those running hdfs, then ideally the regionserver for a particular region should be running on the same server as hosts a region replica.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2138) [hbase] Master should allocate
regions to regionservers based upon data locality and rack awareness
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman updated HADOOP-2138:
----------------------------------
Priority: Minor (was: Major)
Summary: [hbase] Master should allocate regions to regionservers based upon data locality and rack awareness (was: [hbase] Master should allocate regions to the regionserver hosting the region data where possible)
Downgrading priority because we should leverage Hadoop's rack awareness where possible, and there is a lot of work left to do (in Hadoop) before we can
> [hbase] Master should allocate regions to regionservers based upon data locality and rack awareness
> ---------------------------------------------------------------------------------------------------
>
> Key: HADOOP-2138
> URL: https://issues.apache.org/jira/browse/HADOOP-2138
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
> Priority: Minor
>
> Currently, regions are assigned regionservers based off a basic loading attribute. A factor to include in the assignment calcuation is the location of the region in hdfs; i.e. servers hosting region replicas. If the cluster is such that regionservers are being run on the same nodes as those running hdfs, then ideally the regionserver for a particular region should be running on the same server as hosts a region replica.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.