You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Lokesh Jain (JIRA)" <ji...@apache.org> on 2019/01/17 07:48:00 UTC
[jira] [Commented] (HDDS-981) Block allocation should involve
pipeline selection and then container selection
[ https://issues.apache.org/jira/browse/HDDS-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744763#comment-16744763 ]
Lokesh Jain commented on HDDS-981:
----------------------------------
Other issues not addressed in the patch -
# Synchronization might be required in ContainerStateManager#getMatchingContainer to avoid multiple threads using the same container.
# Need to automatically allocate containers to have configured number of containers with atleast some free space. For a busy cluster it is possible to have many clients receiving blocks on the same set of containers. It is essential to have a strategy where ClosedContainerExceptions can be minimized during writes.
> Block allocation should involve pipeline selection and then container selection
> -------------------------------------------------------------------------------
>
> Key: HDDS-981
> URL: https://issues.apache.org/jira/browse/HDDS-981
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: SCM
> Reporter: Lokesh Jain
> Assignee: Lokesh Jain
> Priority: Major
> Attachments: HDDS-981.001.patch
>
>
> Currently SCM maintains a list of preallocated containers and allocates blocks from these containers. This approach does not work well with dynamics of the cluster where new nodes are being added and pipelines are destroyed. New containers are not created until all the preallocated containers are exhausted.
> The Jira aims to establish a criteria in block allocation where first a pipeline is selected amongst the available pipelines and then a container is selected in that pipeline. In order to handle the dynamics of the cluster a fixed interval pipeline creator job can be launched which creates pipelines in the system.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org