You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@inlong.apache.org by GitBox <gi...@apache.org> on 2022/12/26 13:11:08 UTC

[GitHub] [inlong] featzhang opened a new issue, #7072: [Feature][Manager] Resource adaptive adjustment for Hudi

featzhang opened a new issue, #7072:
URL: https://github.com/apache/inlong/issues/7072

   ### Description
   
   Hudi flink jobs often have unreasonable resource allocation. Too much allocation will lead to a waste of resources, and too little will lead to back pressure or OOM.
   
   When allocating resources, you first need to determine the concurrency of the source side to ensure that there is no data backlog upstream when reading. Here is a general configuration situation, such as partitioning by day, with about 15 billion data per day, and about 50 concurrent configurations. Other data volumes can be converted appropriately.
   
   After determining the concurrency on the source side, you can configure the concurrency of write according to the ratio of 1:1.5 or 1:2.
   
   ### Use case
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes, I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@inlong.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Feature][Manager][Sort] Resource adaptive adjustment for Hudi [inlong]

Posted by "dockerzhang (via GitHub)" <gi...@apache.org>.
dockerzhang closed issue #7072: [Feature][Manager][Sort] Resource adaptive adjustment for Hudi
URL: https://github.com/apache/inlong/issues/7072


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@inlong.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [inlong] github-actions[bot] commented on issue #7072: [Feature][Manager][Sort] Resource adaptive adjustment for Hudi

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7072:
URL: https://github.com/apache/inlong/issues/7072#issuecomment-1445248870

   This issue is stale because it has been open for 60 days with no activity.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org