You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pegasus.apache.org by GitBox <gi...@apache.org> on 2022/01/20 06:38:47 UTC
[GitHub] [incubator-pegasus] hycdong opened a new issue #886: enhancement(bulk_load): support disk-level concurrent ingesting count
hycdong opened a new issue #886:
URL: https://github.com/apache/incubator-pegasus/issues/886
In current implementation, meta server control max concurrent ingestion partition count, for example, if max_concurrent_count is 4, meta server will send 4 ingestion request to replica servers, and won't send the fifth request until previous ingestion succeed or failed. Current restriction is precise enough, which can not control concurrent ingesting count of one node, we need to provide another percise plan, a disk-level concurrent ingesting count.
Meta server adds `ingestion_context` to store ingesting partitions, including its node address and disk info. The new plan provides the node max ingesting count and disk max ingesting count. Before sending ingestion request, meta server will check if node and disk max count will be exceed after sending the request, if will not exceed, the request will be sent. Besides, when ingestion succeed or failed, the content of ingestion_context will be reset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] hycdong closed issue #886: enhancement(bulk_load): support disk-level concurrent ingesting count
Posted by GitBox <gi...@apache.org>.
hycdong closed issue #886:
URL: https://github.com/apache/incubator-pegasus/issues/886
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org