You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/06/24 11:18:06 UTC
[GitHub] [ozone] sadanand48 opened a new pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
sadanand48 opened a new pull request #2363:
URL: https://github.com/apache/ozone/pull/2363
## What changes were proposed in this pull request?
Increase default container report interval to 60 mins and also increased pipeline report interval.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5382
## How was this patch tested?
only a config change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi commented on pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on pull request #2363:
URL: https://github.com/apache/ozone/pull/2363#issuecomment-869478970
@sadanand48 , can you elaborate a bit why pipeline report interval is changed to 60m? If I remember correctly, today, Pipeline Manager will use this pipeline report interval as the timeout to wait for a pipeline be ready for serve wright.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 commented on pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
Posted by GitBox <gi...@apache.org>.
JacksonYao287 commented on pull request #2363:
URL: https://github.com/apache/ozone/pull/2363#issuecomment-871917187
maybe for now , we could just only change container full report interval , which is a heavy burden of scm.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi commented on pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on pull request #2363:
URL: https://github.com/apache/ozone/pull/2363#issuecomment-888824084
The last patch LGTM, +1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bshashikant commented on pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
Posted by GitBox <gi...@apache.org>.
bshashikant commented on pull request #2363:
URL: https://github.com/apache/ozone/pull/2363#issuecomment-873980805
> @sadanand48 , can you elaborate a bit why pipeline report interval is changed to 60m? If I remember correctly, today, Pipeline Manager will use this pipeline report interval as the timeout to wait for a pipeline be ready for serve wright.
I think, the pipeline report is sent on heartbeat as soon as a pipeline is created or pipeline is closed and destroyed. There is no dependency on the pipeline report interval as such for opening up the pipeline for write. Full report intervals can be sent a higher interval.
cc ~ @nandakumar131
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bshashikant commented on pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
Posted by GitBox <gi...@apache.org>.
bshashikant commented on pull request #2363:
URL: https://github.com/apache/ozone/pull/2363#issuecomment-873980805
> @sadanand48 , can you elaborate a bit why pipeline report interval is changed to 60m? If I remember correctly, today, Pipeline Manager will use this pipeline report interval as the timeout to wait for a pipeline be ready for serve wright.
I think, the pipeline report is sent on heartbeat as soon as a pipeline is created or pipeline is closed and destroyed. There is no dependency on the pipeline report interval as such for opening up the pipeline for write. Full report intervals can be sent a higher interval.
cc ~ @nandakumar131
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bshashikant edited a comment on pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
Posted by GitBox <gi...@apache.org>.
bshashikant edited a comment on pull request #2363:
URL: https://github.com/apache/ozone/pull/2363#issuecomment-868187946
@guihecheng , during scale testing of ozone with 350k+ containers and nearly 1 million replica reports it was observed that, there is a sudden burst in scm heap usage . In hdfs, the fill block report interval is 6 hours by default and in between, there are incremental bock reports. Similarly, there are incremental reports in SCM . Setting the full container report interval to 1 hour make things quite stable as determined from tests and 60s for full report seems very aggressive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bshashikant commented on pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
Posted by GitBox <gi...@apache.org>.
bshashikant commented on pull request #2363:
URL: https://github.com/apache/ozone/pull/2363#issuecomment-868187946
@guihecheng , during scale testing if ozone with 350k+ containers and nearly 1 million replica reports it was observed that, there is a sudden burst in scm heap usage . In hdfs, the fill block report interval is 6 hours by default and in between, there are incremental bock reports. Similarly, there are incremental reports in SCM . Setting the full container report interval to 1 hour make things quite stable as determined from tests and 60s for full report seems very aggressive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bshashikant commented on pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
Posted by GitBox <gi...@apache.org>.
bshashikant commented on pull request #2363:
URL: https://github.com/apache/ozone/pull/2363#issuecomment-868188557
@sadanand48 , please change the default container and pipeline report interval to 60s in acceotance tests and in MiniOzoneCluster so that there are no random/intermittent failures bcoz of this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] guihecheng commented on pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
Posted by GitBox <gi...@apache.org>.
guihecheng commented on pull request #2363:
URL: https://github.com/apache/ozone/pull/2363#issuecomment-868241183
Thanks @bshashikant for the explanation, this is very useful pr for a production deployment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bshashikant commented on pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
Posted by GitBox <gi...@apache.org>.
bshashikant commented on pull request #2363:
URL: https://github.com/apache/ozone/pull/2363#issuecomment-878145249
@nandakumar131 , can you also have a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] guihecheng commented on pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
Posted by GitBox <gi...@apache.org>.
guihecheng commented on pull request #2363:
URL: https://github.com/apache/ozone/pull/2363#issuecomment-868164637
Hi @sadanand48 , sorry to disturb, but I happen to see this and can't help wonder why the default report intervals are changed? I can't find any descriptions in the related jira either, could you please explain a bit? thanks~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] mukul1987 merged pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
Posted by GitBox <gi...@apache.org>.
mukul1987 merged pull request #2363:
URL: https://github.com/apache/ozone/pull/2363
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] jojochuang commented on pull request #2363: HDDS-5382. Increase default container report interval to 60 mins
Posted by GitBox <gi...@apache.org>.
jojochuang commented on pull request #2363:
URL: https://github.com/apache/ozone/pull/2363#issuecomment-887180018
I support increasing the default interval. @ChenSammi feel free to let us know if something is still holding back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org