You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "adoroszlai (via GitHub)" <gi...@apache.org> on 2023/04/27 17:48:02 UTC
[GitHub] [ozone] adoroszlai opened a new pull request, #4627: HDDS-8471. ReplicationManager: Avoid re-queuing duplicate message for under / over replicated containers
adoroszlai opened a new pull request, #4627:
URL: https://github.com/apache/ozone/pull/4627
## What changes were proposed in this pull request?
Make over/under-replicated processors use a single instance of `ReplicationQueue` for each iteration, even if `ReplicationManager` replaces it with a new queue concurrently.
https://issues.apache.org/jira/browse/HDDS-8471
## How was this patch tested?
CI:
https://github.com/adoroszlai/hadoop-ozone/actions/runs/4821912584
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel commented on pull request #4627: HDDS-8471. ReplicationManager: Avoid re-queuing duplicate message for under / over replicated containers
Posted by "sodonnel (via GitHub)" <gi...@apache.org>.
sodonnel commented on PR #4627:
URL: https://github.com/apache/ozone/pull/4627#issuecomment-1527273493
> We can simply make each sub-queue in `ReplicationQueue` thread-safe by wrapping in `Queues.synchronizedQueue` (from Guava).
Sounds good to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel commented on pull request #4627: HDDS-8471. ReplicationManager: Avoid re-queuing duplicate message for under / over replicated containers
Posted by "sodonnel (via GitHub)" <gi...@apache.org>.
sodonnel commented on PR #4627:
URL: https://github.com/apache/ozone/pull/4627#issuecomment-1527248279
The lock that serialized access to the queue is removed in this PR. Right now, we only have 2 threads that access the queue - the under-rep processor and the over-rep processor. Each of them runs concurrently, and generally accesses only the underRep queue or the overRep queue, so it is probably safe. I wonder if we should make the queue object thread safe via synchronised methods or a lock to ensure that only 1 thread can modify it at a time. It's hard to know how the calling code might change in the future which might result in concurrency issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #4627: HDDS-8471. ReplicationManager: Avoid re-queuing duplicate message for under / over replicated containers
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #4627:
URL: https://github.com/apache/ozone/pull/4627#issuecomment-1527269389
Thanks @sodonnel for the review.
We can simply make each sub-queue in `ReplicationQueue` thread-safe by wrapping in `Queues.synchronizedQueue` (from Guava).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai merged pull request #4627: HDDS-8471. Ensure replication processors use a single queue for each iteration
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #4627:
URL: https://github.com/apache/ozone/pull/4627
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel commented on pull request #4627: HDDS-8471. ReplicationManager: Avoid re-queuing duplicate message for under / over replicated containers
Posted by "sodonnel (via GitHub)" <gi...@apache.org>.
sodonnel commented on PR #4627:
URL: https://github.com/apache/ozone/pull/4627#issuecomment-1527258458
Changes LGTM - I just wonder if we synchronize on the ReplicationQueue methods to make it safe for changes in the future - what do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org