[jira] [Created] (GOBBLIN-1604) Throw Allocation Error if all attempts to allocate workunits exceeds resource maximums

["William Lo (Jira)" <ji...@apache.org>]

William Lo created GOBBLIN-1604:
-----------------------------------

             Summary: Throw Allocation Error if all attempts to allocate workunits exceeds resource maximums
                 Key: GOBBLIN-1604
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1604
             Project: Apache Gobblin
          Issue Type: Task
            Reporter: William Lo


In distcp, if the size or number of files allocated by the RequestAllocator during workunit collection is too large, then Gobblin will log the error:
{code:java}
// item does not fit even in empty pool
log.warn(String.format("Request %s is larger than the available resource pool. If the pool is not expanded, "
        + "it will never be selected. Request: %s.", newElement.getT(),
{code}

The job will still be marked as COMMITTED/SUCCESSFUL state, when it should really be partial success of FAIL if there are no workunits collected, to make the failure more visible to users.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)