You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/11/25 09:27:38 UTC
[GitHub] [flink-table-store] tsreaper opened a new pull request, #403: [FLINK-30205] Modify compact interface for TableWrite and FileStoreWrite to support normal compaction in Table Store
tsreaper opened a new pull request, #403:
URL: https://github.com/apache/flink-table-store/pull/403
Currently the compact interface in `TableWrite` and `FileStoreWrite` can only trigger full compaction. However a separated compact job should not only perform full compaction, but also perform normal compaction once in a while, just like what the current Table Store sinks do.
We need to modify compact interface for `TableWrite` and `FileStoreWrite` to support normal compaction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-table-store] JingsongLi merged pull request #403: [FLINK-30205] Modify compact interface for TableWrite and FileStoreWrite to support normal compaction in Table Store
Posted by GitBox <gi...@apache.org>.
JingsongLi merged PR #403:
URL: https://github.com/apache/flink-table-store/pull/403
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-table-store] JingsongLi commented on a diff in pull request #403: [FLINK-30205] Modify compact interface for TableWrite and FileStoreWrite to support normal compaction in Table Store
Posted by GitBox <gi...@apache.org>.
JingsongLi commented on code in PR #403:
URL: https://github.com/apache/flink-table-store/pull/403#discussion_r1033075278
##########
flink-table-store-core/src/main/java/org/apache/flink/table/store/file/utils/RecordWriter.java:
##########
@@ -34,10 +34,12 @@
void write(T record) throws Exception;
/**
- * Compact all files related to the writer. Note that compaction process is only submitted and
- * may not be completed when the method returns.
+ * Compact files related to the writer. Note that compaction process is only submitted and may
+ * not be completed when the method returns.
+ *
+ * @param fullCompaction whether to trigger full compaction or just normal compaction
Review Comment:
I think there are two things:
- Is this compaction full or normal.
- Should we wait latest compaction.
I think we can separate these two things.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-table-store] tsreaper commented on a diff in pull request #403: [FLINK-30205] Modify compact interface for TableWrite and FileStoreWrite to support normal compaction in Table Store
Posted by GitBox <gi...@apache.org>.
tsreaper commented on code in PR #403:
URL: https://github.com/apache/flink-table-store/pull/403#discussion_r1033078213
##########
flink-table-store-core/src/main/java/org/apache/flink/table/store/file/utils/RecordWriter.java:
##########
@@ -34,10 +34,12 @@
void write(T record) throws Exception;
/**
- * Compact all files related to the writer. Note that compaction process is only submitted and
- * may not be completed when the method returns.
+ * Compact files related to the writer. Note that compaction process is only submitted and may
+ * not be completed when the method returns.
+ *
+ * @param fullCompaction whether to trigger full compaction or just normal compaction
Review Comment:
From the point of a separated compact job, we should always wait for last compaction to complete. Otherwise new changes may never be compacted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-table-store] JingsongLi commented on a diff in pull request #403: [FLINK-30205] Modify compact interface for TableWrite and FileStoreWrite to support normal compaction in Table Store
Posted by GitBox <gi...@apache.org>.
JingsongLi commented on code in PR #403:
URL: https://github.com/apache/flink-table-store/pull/403#discussion_r1033177161
##########
flink-table-store-core/src/main/java/org/apache/flink/table/store/file/utils/RecordWriter.java:
##########
@@ -34,10 +34,12 @@
void write(T record) throws Exception;
/**
- * Compact all files related to the writer. Note that compaction process is only submitted and
- * may not be completed when the method returns.
+ * Compact files related to the writer. Note that compaction process is only submitted and may
+ * not be completed when the method returns.
+ *
+ * @param fullCompaction whether to trigger full compaction or just normal compaction
Review Comment:
You are right
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org