You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by GitBox <gi...@apache.org> on 2022/08/30 23:39:34 UTC
[GitHub] [gobblin] Will-Lo commented on a diff in pull request #3544: [GOBBLIN-1689] Decouple compiler from scheduler in warm standby mode
Will-Lo commented on code in PR #3544:
URL: https://github.com/apache/gobblin/pull/3544#discussion_r959018143
##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/spec_catalog/FlowCatalog.java:
##########
@@ -383,10 +387,17 @@ public Map<String, AddSpecResponse> put(Spec spec, boolean triggerListener) thro
return responseMap;
}
}
- AddSpecResponse<String> schedulerResponse = responseMap.getOrDefault(ServiceConfigKeys.GOBBLIN_SERVICE_JOB_SCHEDULER_LISTENER_CLASS, new AddSpecResponse<>(null));
+ AddSpecResponse<String> compileResponse;
+ if (isWarmStandbyEnabled) {
+ compileResponse = responseMap.getOrDefault(ServiceConfigKeys.GOBBLIN_ORCHESTRATOR_LISTENER_CLASS, new AddSpecResponse<>(null));
+ //todo: do we check quota here? or in compiler? Quota manager need dag to check quota which is not accessable from this class
Review Comment:
Quota check should only be enabled for adhoc flows or flows who are set to run immediately, otherwise it should be safe to store. We can't block users from adding flows to be scheduled in the future even if their current quota is overrun.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org