You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/01/31 13:14:10 UTC
[GitHub] [hudi] nsivabalan commented on a change in pull request #2496: [HUDI-1554] Introduced buffering for streams in HUDI.
nsivabalan commented on a change in pull request #2496:
URL: https://github.com/apache/hudi/pull/2496#discussion_r567422859
##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java
##########
@@ -931,6 +941,21 @@ public int getMetadataCleanerCommitsRetained() {
return Integer.parseInt(props.getProperty(HoodieMetadataConfig.CLEANER_COMMITS_RETAINED_PROP));
}
+ /**
+ * File system IO buffering configs.
+ */
+ public boolean isFileIOBufferingEnabled() {
+ return Boolean.parseBoolean(props.getProperty(FILE_IO_BUFFER_ENABLED, DEFAULT_FILE_IO_BUFFER_ENABLED));
+ }
+
+ public int getFileIOBufferiMinSize() {
Review comment:
minor typo. "g" for buffering is missing
##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java
##########
@@ -1193,6 +1218,21 @@ public Builder withMergeAllowDuplicateOnInserts(boolean routeInsertsToNewFiles)
return this;
}
+ public Builder withFileIOBufferEnabled(boolean enabled) {
+ props.setProperty(FILE_IO_BUFFER_ENABLED, String.valueOf(enabled));
+ return this;
+ }
+
+ public Builder withFileIOBufferMinSize(int sizeInBytes) {
Review comment:
can we please try to be uniform. for getter methods, we have suffixed "ing" for buffer, but here we haven't. was it a conscious decision?
##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java
##########
@@ -931,6 +941,21 @@ public int getMetadataCleanerCommitsRetained() {
return Integer.parseInt(props.getProperty(HoodieMetadataConfig.CLEANER_COMMITS_RETAINED_PROP));
}
+ /**
+ * File system IO buffering configs.
+ */
+ public boolean isFileIOBufferingEnabled() {
+ return Boolean.parseBoolean(props.getProperty(FILE_IO_BUFFER_ENABLED, DEFAULT_FILE_IO_BUFFER_ENABLED));
+ }
+
+ public int getFileIOBufferiMinSize() {
+ return Integer.parseInt(props.getProperty(MIN_FILE_IO_BUFFER_SIZE, DEFAULT_MIN_FILE_IO_BUFFER_SIZE_BYTES));
+ }
+
+ public int getDataFileIOBufferiMinSize() {
Review comment:
same here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org