You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/08/25 09:51:08 UTC
[GitHub] [ozone] JacksonYao287 commented on a change in pull request #2569: HDDS-5655. SCM terminates when allocatecontainer happens on closed pipeline.
JacksonYao287 commented on a change in pull request #2569:
URL: https://github.com/apache/ozone/pull/2569#discussion_r695588583
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineManagerImpl.java
##########
@@ -69,7 +68,7 @@
LoggerFactory.getLogger(PipelineManagerImpl.class);
// Limit the number of on-going ratis operation to be 1.
- private final Lock lock;
+ private final ReentrantReadWriteLock lock;
Review comment:
`PipelineStateManagerV2Impl` has an internal lock too , write options , such as `removePipeline` ,will be protected by this lock.
```
public void removePipeline(HddsProtos.PipelineID pipelineIDProto)
throws IOException {
lock.writeLock().lock();
try {
.............
} finally {
lock.writeLock().unlock();
}
}
```
can we take it into to account?
since `PipelineStateManagerV2Impl` is a private member of `PipelineManagerImpl`, `PipelineStateManagerV2Impl` can be totally protected by `PipelineManagerImpl`. if we add a lock to `PipelineManagerImpl`, can we remove the internal lock of `PipelineStateManagerV2Impl`?
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerManagerImpl.java
##########
@@ -176,33 +176,64 @@ public ContainerInfo getContainer(final ContainerID id)
public ContainerInfo allocateContainer(
final ReplicationConfig replicationConfig, final String owner)
throws IOException {
+ // Acquire pipeline manager lock, to avoid any updates to pipeline
+ // while allocate container happens. This is to avoid scenario like
+ // mentioned in HDDS-5655.
+ pipelineManager.acquireReadLock();
lock.lock();
+ List<Pipeline> pipelines;
+ Pipeline pipeline;
+ ContainerInfo containerInfo = null;
try {
- final List<Pipeline> pipelines = pipelineManager
+ pipelines = pipelineManager
.getPipelines(replicationConfig, Pipeline.PipelineState.OPEN);
+ if (!pipelines.isEmpty()) {
+ pipeline = pipelines.get(random.nextInt(pipelines.size()));
+ containerInfo = createContainer(pipeline, owner);
+ }
+ } finally {
+ lock.unlock();
+ pipelineManager.releaseReadLock();
+ }
- final Pipeline pipeline;
- if (pipelines.isEmpty()) {
- try {
- pipeline = pipelineManager.createPipeline(replicationConfig);
- pipelineManager.waitPipelineReady(pipeline.getId(), 0);
- } catch (IOException e) {
- scmContainerManagerMetrics.incNumFailureCreateContainers();
+ if (pipelines.isEmpty()) {
+ try {
+ pipeline = pipelineManager.createPipeline(replicationConfig);
+ pipelineManager.waitPipelineReady(pipeline.getId(), 0);
+ } catch (IOException e) {
+ scmContainerManagerMetrics.incNumFailureCreateContainers();
+ throw new IOException("Could not allocate container. Cannot get any" +
+ " matching pipeline for replicationConfig: " + replicationConfig
+ + ", State:PipelineState.OPEN", e);
+ }
+ pipelineManager.acquireReadLock();
+ lock.lock();
+ try {
+ pipelines = pipelineManager
+ .getPipelines(replicationConfig, Pipeline.PipelineState.OPEN);
+ if (!pipelines.isEmpty()) {
+ pipeline = pipelines.get(random.nextInt(pipelines.size()));
+ containerInfo = createContainer(pipeline, owner);
+ } else {
throw new IOException("Could not allocate container. Cannot get any" +
" matching pipeline for replicationConfig: " + replicationConfig
- + ", State:PipelineState.OPEN", e);
+ + ", State:PipelineState.OPEN");
}
- } else {
- pipeline = pipelines.get(random.nextInt(pipelines.size()));
- }
- final ContainerInfo containerInfo = allocateContainer(pipeline, owner);
- if (LOG.isTraceEnabled()) {
- LOG.trace("New container allocated: {}", containerInfo);
+ } finally {
+ lock.unlock();
+ pipelineManager.releaseReadLock();
Review comment:
can you refactor these code to some like
```
pipelineManager.acquireReadLock();
lock.lock();
try {
pipelines = pipelineManager
.getPipelines(replicationConfig, Pipeline.PipelineState.OPEN);
if (pipelines.isEmpty()){
.............
}else{
............
}
} finally{
lock.unlock();
pipelineManager.releaseReadLock();
}
```
i think this will remove some redundant code and the logic here will be more clear
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org