You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 11:45:45 UTC
[I] Clean action sometimes falls back to spark.default.parallelism set in the env [hudi]
hudi-bot opened a new issue, #16871:
URL: https://github.com/apache/hudi/issues/16871
Rarely, we notice there are 1024 (spark.default.parallelism) tasks spinning up for clean actions.
for eg, if we try to ingest very small no of records say just 1, clean action executor spins up 1024 tasks even though the clean parallelism is set to small value.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-9067
- Type: Improvement
- Fix version(s):
- 1.1.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Clean action sometimes falls back to spark.default.parallelism set in the env [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua closed issue #16871: Clean action sometimes falls back to spark.default.parallelism set in the env
URL: https://github.com/apache/hudi/issues/16871
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Clean action sometimes falls back to spark.default.parallelism set in the env [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on issue #16871:
URL: https://github.com/apache/hudi/issues/16871#issuecomment-3630285640
### Linked PR(s)
- https://github.com/apache/hudi/pull/12874
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org