You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/11/18 06:41:36 UTC
[GitHub] [flink-table-store] tsreaper commented on a diff in pull request #388: [FLINK-29232] File store continuous reading support from_timestamp scan mode
tsreaper commented on code in PR #388:
URL: https://github.com/apache/flink-table-store/pull/388#discussion_r1026074919
##########
flink-table-store-connector/src/main/java/org/apache/flink/table/store/connector/source/FlinkSourceBuilder.java:
##########
@@ -150,20 +153,20 @@ private FileStoreSource buildFileSource(boolean isContinuous, boolean continuous
LogStartupMode startupMode = conf.get(LOG_SCAN);
if (logSourceProvider == null) {
- return buildFileSource(true, startupMode == LogStartupMode.LATEST);
+ return buildFileSource(true, startupMode, conf.get(LOG_SCAN_TIMESTAMP_MILLS));
} else {
if (startupMode != LogStartupMode.FULL) {
return logSourceProvider.createSource(null);
}
return HybridSource.<RowData, StaticFileStoreSplitEnumerator>builder(
- buildFileSource(false, false))
+ buildFileSource(false, startupMode, null))
.addSource(
new LogHybridSourceFactory(logSourceProvider),
Boundedness.CONTINUOUS_UNBOUNDED)
.build();
}
} else {
- return buildFileSource(false, false);
+ return buildFileSource(false, LogStartupMode.FULL, null);
Review Comment:
Assert that startup mode is `FULL`.
##########
flink-table-store-connector/src/main/java/org/apache/flink/table/store/connector/source/FlinkSourceBuilder.java:
##########
@@ -150,20 +153,20 @@ private FileStoreSource buildFileSource(boolean isContinuous, boolean continuous
LogStartupMode startupMode = conf.get(LOG_SCAN);
if (logSourceProvider == null) {
- return buildFileSource(true, startupMode == LogStartupMode.LATEST);
+ return buildFileSource(true, startupMode, conf.get(LOG_SCAN_TIMESTAMP_MILLS));
Review Comment:
In previous table store versions, when a table property starts with `log.`, it is meant to control the behavior of Kafka. Now you're using these properties to control both Kafka and file store.
You might need to clarify some concepts. What is a "log"? Is it only about Kafka? Shall we also consider changelog produced by file store as log?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org