You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "lokeshj1703 (via GitHub)" <gi...@apache.org> on 2026/07/01 11:35:33 UTC
Re: [PR] fix: S3 incremental source should impose limit on num of files consumed per batch [hudi]
lokeshj1703 commented on code in PR #18076:
URL: https://github.com/apache/hudi/pull/18076#discussion_r3505510979
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/config/CloudSourceConfig.java:
##########
@@ -166,6 +166,15 @@ public class CloudSourceConfig extends HoodieConfig {
.sinceVersion("0.14.1")
.withDocumentation("specify this value in bytes, to coalesce partitions of source dataset not greater than specified limit");
+ public static final ConfigProperty<Long> SOURCE_MAX_FILES_PER_SYNC = ConfigProperty
+ .key(STREAMER_CONFIG_PREFIX + "source.cloud.data.max.files.per.sync")
+ .defaultValue(10_000_000L)
+ .markAdvanced()
+ .sinceVersion("1.4.0")
Review Comment:
`sinceVersion("1.4.0")` should be `"1.3.0"` — the current pom version is `1.3.0-SNAPSHOT`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org