You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by GitBox <gi...@apache.org> on 2021/04/26 22:52:24 UTC
[GitHub] [samza] cameronlee314 opened a new pull request #1494: SAMZA-2648: Add simple placeholder FaultDomainManager for when YARN is not used
cameronlee314 opened a new pull request #1494:
URL: https://github.com/apache/samza/pull/1494
Issues: There is currently only one concrete `FaultDomainManager` implementation, and it is for YARN usage. It is also specified as the default in code. However, for non-YARN environments, we shouldn't include YARN dependencies, so we need another concrete implementation. For now, we can just add a very simple implementation as a placeholder.
Changes: Added `SingleFaultDomainManager` which is a simple impl of `FaultDomainManager` which only has a single fault domain. This isn't wired in anywhere, but a job can configure it as the `FaultDomainManager` if desirable.
Tests: Ran a simple test job on Kubernetes using this class (also excluded all code from `samza-yarn`). This won't work if standby containers are enabled, but it can still be a useful placeholder when standby containers are not needed.
API changes and usage/upgrade instructions:
Set the config `cluster-manager.fault-domain-manager.factory` to `org.apache.samza.clustermanager. SingleFaultDomainManagerFactory` to use this new fault domain manager.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [samza] cameronlee314 commented on pull request #1494: SAMZA-2648: Add simple placeholder FaultDomainManager for when YARN is not used
Posted by GitBox <gi...@apache.org>.
cameronlee314 commented on pull request #1494:
URL: https://github.com/apache/samza/pull/1494#issuecomment-828842377
> What do you mean by this line in your description: `This won't work if standby containers are enabled, but it can still be a useful placeholder when standby containers are not needed.`?
> For jobs to be fault-domain aware, they need to turn on this config specifically: `cluster-manager.fault-domain-aware.standby.enabled`. Not all jobs with standby containers are fault domain aware.
Sorry, my comment wasn't correct. Thanks for pointing out that config. I updated it to `Any features which depend on fault domain awareness won't work (e.g. using standby containers with cluster-manager.fault-domain-aware.standby.enabled = true won't work), but it can still be a useful placeholder when standby containers are not needed.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [samza] PawasChhokra commented on pull request #1494: SAMZA-2648: Add simple placeholder FaultDomainManager for when YARN is not used
Posted by GitBox <gi...@apache.org>.
PawasChhokra commented on pull request #1494:
URL: https://github.com/apache/samza/pull/1494#issuecomment-828834605
What do you mean by this line in your description: `This won't work if standby containers are enabled, but it can still be a useful placeholder when standby containers are not needed.`?
For jobs to be fault-domain aware, they need to turn on this config specifically: `cluster-manager.fault-domain-aware.standby.enabled`. Not all jobs with standby containers are fault domain aware.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [samza] cameronlee314 merged pull request #1494: SAMZA-2648: Add simple placeholder FaultDomainManager for when YARN is not used
Posted by GitBox <gi...@apache.org>.
cameronlee314 merged pull request #1494:
URL: https://github.com/apache/samza/pull/1494
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org