You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by GitBox <gi...@apache.org> on 2022/02/08 21:29:25 UTC
[GitHub] [gobblin] ZihanLi58 commented on a change in pull request #3461: [GOBBLIN-1604] Throw exception if there are no allocated requests due to lack of res…
ZihanLi58 commented on a change in pull request #3461:
URL: https://github.com/apache/gobblin/pull/3461#discussion_r802073840
##########
File path: gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/CopySource.java
##########
@@ -291,6 +292,20 @@ private void submitUnfulfilledRequestEventsHelper(List<FileSet<CopyEntity>> file
}
}
+ void failJobIfAllRequestsRejected(RequestAllocator<FileSet<CopyEntity>> allocator,
+ Iterator<FileSet<CopyEntity>> allocatedRequests) throws IOException {
+ // TODO: we should set job as partial success if there is a mix of allocated requests and rejections
Review comment:
I think this partial success is seen frequently as well, worth to address.
##########
File path: gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/CopySource.java
##########
@@ -291,6 +292,20 @@ private void submitUnfulfilledRequestEventsHelper(List<FileSet<CopyEntity>> file
}
}
+ void failJobIfAllRequestsRejected(RequestAllocator<FileSet<CopyEntity>> allocator,
+ Iterator<FileSet<CopyEntity>> allocatedRequests) throws IOException {
+ // TODO: we should set job as partial success if there is a mix of allocated requests and rejections
+ if (PriorityIterableBasedRequestAllocator.class.isAssignableFrom(allocator.getClass())) {
+ PriorityIterableBasedRequestAllocator<FileSet<CopyEntity>> priorityIterableBasedRequestAllocator =
+ (PriorityIterableBasedRequestAllocator<FileSet<CopyEntity>>) allocator;
+ // If there are no allocated items and there items exceeding the available resources, then we can infer all items exceed resources
Review comment:
If there are no allocated items and there "are" items exceeding the available resources, then we can infer all items exceed resources?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org