[jira] [Created] (GOBBLIN-1654) Add capacity floor to avoid aggressively requesting resource and small files.

["Zihan Li (Jira)" <ji...@apache.org>]

Zihan Li created GOBBLIN-1654:
---------------------------------

             Summary: Add capacity floor to avoid aggressively requesting resource and small files.
                 Key: GOBBLIN-1654
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1654
             Project: Apache Gobblin
          Issue Type: Improvement
            Reporter: Zihan Li


No we calculate the capacity based on the consumer rate during peak hour, but if there is record with super bad schema during that time, our consumer rate will be super low and even after we catch up, we will not able to release the resources because of the low consumer rate. So want provide a config to set the minimum consumer rate and avoid abusing resources ad small files



--
This message was sent by Atlassian Jira
(v8.20.7#820007)