You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "guohao-rosicky (via GitHub)" <gi...@apache.org> on 2023/06/16 06:52:53 UTC
[GitHub] [ozone] guohao-rosicky opened a new pull request, #4912: HDDS-8855. Filter owner when getting ec container
guohao-rosicky opened a new pull request, #4912:
URL: https://github.com/apache/ozone/pull/4912
## What changes were proposed in this pull request?
Filter owner when getting ec container
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8855
## How was this patch tested?
UT: TestWritableECContainerProvider.java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #4912: HDDS-8855. Filter owner when getting ec container
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #4912:
URL: https://github.com/apache/ozone/pull/4912#issuecomment-1603796521
@guohao-rosicky @ivandika3
From HDDS-8874 it looks like `owner` metadata, designed before OM HA, is incorrectly set when using OM HA. Thus, the information currently stored is basically useless. Until that is fixed, this change would introduce further problems as @sodonnel described. This filtering issue should be fixed after or as part of the OM HA owner problem, certainly not sooner.
Therefore I propose closing this PR but keeping HDDS-8855 open.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] guohao-rosicky commented on pull request #4912: HDDS-8855. Filter owner when getting ec container
Posted by "guohao-rosicky (via GitHub)" <gi...@apache.org>.
guohao-rosicky commented on PR #4912:
URL: https://github.com/apache/ozone/pull/4912#issuecomment-1594199577
I changed the code so that it doesn't close the pipeline. hi @sodonnel PTAL. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] guohao-rosicky commented on pull request #4912: HDDS-8855. Filter owner when getting ec container
Posted by "guohao-rosicky (via GitHub)" <gi...@apache.org>.
guohao-rosicky commented on PR #4912:
URL: https://github.com/apache/ozone/pull/4912#issuecomment-1605894270
@adoroszlai @ivandika3
As @sodonnel said, we should follow up with a plan to support the ec container to the owner, just as the ratis container does now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] guohao-rosicky commented on pull request #4912: HDDS-8855. Filter owner when getting ec container
Posted by "guohao-rosicky (via GitHub)" <gi...@apache.org>.
guohao-rosicky commented on PR #4912:
URL: https://github.com/apache/ozone/pull/4912#issuecomment-1594222525
I accidentally closed this pr(https://github.com/apache/ozone/pull/4905), let's continue the discussion with this pr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel commented on pull request #4912: HDDS-8855. Filter owner when getting ec container
Posted by "sodonnel (via GitHub)" <gi...@apache.org>.
sodonnel commented on PR #4912:
URL: https://github.com/apache/ozone/pull/4912#issuecomment-1594448710
I am still not sure about this change. If the cluster is started, and owner1 starts writing EC data, then it will create pipelines up to the pipeline limit. Then if owner2 comes in and tries to get a pipeline, it will fail to get one and an error will be returned. Or you could end up with a case where ower1 has most of the pipelines and all writes for owner2 go to a very small number of pipelines.
We are enforcing open container limits at the pipeline level with EC, rather than the container level. You can think of a pipeline and container as the same thing with EC, as they are tightly coupled.
It is still not clear to me how you are using "owner". Where does it come from? How many owners can be seen on a cluster? Is it controlled by OM, or can clients pass different owners freely?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai closed pull request #4912: HDDS-8855. Filter owner when getting ec container
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai closed pull request #4912: HDDS-8855. Filter owner when getting ec container
URL: https://github.com/apache/ozone/pull/4912
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ivandika3 commented on pull request #4912: HDDS-8855. Filter owner when getting ec container
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on PR #4912:
URL: https://github.com/apache/ozone/pull/4912#issuecomment-1596447532
> It is still not clear to me how you are using "owner". Where does it come from? How many owners can be seen on a cluster?
I think I can shed some light on this. The owner is set by the OM HA leader's Node ID (i.e. `OzoneManager#getOmNodeId`) which is set by the Ozone configuration. Hence, within a cluster with N OM nodes, there might be up to N different container owners. I have raised MR regarding this (https://github.com/apache/ozone/pull/4922), maybe we can discuss there as well.
> Is it controlled by OM, or can clients pass different owners freely?
Currently only OM will set the container owner through the `allocateBlock` call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org