You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "SteveYurongSu (via GitHub)" <gi...@apache.org> on 2024/06/17 10:36:21 UTC
[PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
SteveYurongSu opened a new pull request, #12751:
URL: https://github.com/apache/iotdb/pull/12751
As title.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#issuecomment-2174992632
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=12751&metric=new_coverage&view=list) (required ≥ 80%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#issuecomment-2174831712
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=12751&metric=new_coverage&view=list) (required ≥ 80%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#issuecomment-2173131907
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=12751&metric=new_coverage&view=list) (required ≥ 80%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#issuecomment-2173029641
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=12751&metric=new_coverage&view=list) (required ≥ 80%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "SteveYurongSu (via GitHub)" <gi...@apache.org>.
SteveYurongSu commented on code in PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#discussion_r1642831788
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tablet/PipeInsertNodeTabletInsertionEvent.java:
##########
@@ -205,29 +205,36 @@ public boolean isGeneratedByPipe() {
@Override
public boolean mayEventTimeOverlappedWithTimeRange() {
try {
- final InsertNode insertNode = getInsertNode();
+ final InsertNode insertNode = getInsertNodeViaCacheIfPossible();
+ if (Objects.isNull(insertNode)) {
+ return true;
+ }
+
if (insertNode instanceof InsertRowNode) {
final long timestamp = ((InsertRowNode) insertNode).getTime();
return startTime <= timestamp && timestamp <= endTime;
- } else if (insertNode instanceof InsertTabletNode) {
+ }
+
+ if (insertNode instanceof InsertTabletNode) {
final long[] timestamps = ((InsertTabletNode) insertNode).getTimes();
if (Objects.isNull(timestamps) || timestamps.length == 0) {
return false;
}
// We assume that `timestamps` is ordered.
return startTime <= timestamps[timestamps.length - 1] && timestamps[0] <= endTime;
- } else if (insertNode instanceof InsertRowsNode) {
+ }
+
+ if (insertNode instanceof InsertRowsNode) {
for (final InsertRowNode node : ((InsertRowsNode) insertNode).getInsertRowNodeList()) {
final long timestamp = node.getTime();
Review Comment:
Good idea. Applied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#issuecomment-2175407840
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=12751&metric=new_coverage&view=list) (required ≥ 80%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#issuecomment-2173207420
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=12751&metric=new_coverage&view=list) (required ≥ 80%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#issuecomment-2174956807
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=12751&metric=new_coverage&view=list) (required ≥ 80%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "SteveYurongSu (via GitHub)" <gi...@apache.org>.
SteveYurongSu commented on code in PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#discussion_r1642695397
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/extractor/dataregion/realtime/PipeRealtimeDataRegionExtractor.java:
##########
@@ -142,10 +147,33 @@ public void validate(final PipeParameterValidator validator) throws Exception {
EXTRACTOR_END_TIME_KEY,
realtimeDataExtractionEndTime));
}
+ } catch (final PipeParameterNotValidException e) {
+ throw e;
} catch (final Exception e) {
// compatible with the current validation framework
throw new PipeParameterNotValidException(e.getMessage());
}
+
+ final Set<String> sloppyOptionSet =
+ Arrays.stream(
+ parameters
+ .getStringOrDefault(
+ Arrays.asList(
+ EXTRACTOR_REALTIME_LOOSE_RANGE_KEY, SOURCE_REALTIME_LOOSE_RANGE_KEY),
+ EXTRACTOR_REALTIME_LOOSE_RANGE_DEFAULT_VALUE)
+ .split(","))
+ .map(String::trim)
+ .map(String::toLowerCase)
+ .collect(Collectors.toSet());
+ // Avoid empty string
+ sloppyOptionSet.remove("");
+ sloppyTimeRange = sloppyOptionSet.contains(EXTRACTOR_REALTIME_LOOSE_RANGE_TIME_VALUE);
+ sloppyPattern = sloppyOptionSet.contains(EXTRACTOR_REALTIME_LOOSE_RANGE_PATH_VALUE);
+ if (!sloppyOptionSet.isEmpty()) {
Review Comment:
Good catch. Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "VGalaxies (via GitHub)" <gi...@apache.org>.
VGalaxies commented on code in PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#discussion_r1642686394
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/extractor/dataregion/realtime/PipeRealtimeDataRegionExtractor.java:
##########
@@ -142,10 +147,33 @@ public void validate(final PipeParameterValidator validator) throws Exception {
EXTRACTOR_END_TIME_KEY,
realtimeDataExtractionEndTime));
}
+ } catch (final PipeParameterNotValidException e) {
Review Comment:
Also, make changes here https://github.com/apache/iotdb/blob/master/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/extractor/dataregion/historical/PipeHistoricalDataRegionTsFileExtractor.java#L186
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#issuecomment-2173369049
## [Codecov](https://app.codecov.io/gh/apache/iotdb/pull/12751?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `31.50685%` with `50 lines` in your changes missing coverage. Please review.
> Project coverage is 41.75%. Comparing base [(`e6e819a`)](https://app.codecov.io/gh/apache/iotdb/commit/e6e819ad1fbebccefd7d7f05eda95ce192f79f0f?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`5f79e07`)](https://app.codecov.io/gh/apache/iotdb/commit/5f79e07b1ea0c2b5f8c713d229a36b5c2fbcc403?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
| [Files](https://app.codecov.io/gh/apache/iotdb/pull/12751?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...mon/tablet/PipeInsertNodeTabletInsertionEvent.java](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fevent%2Fcommon%2Ftablet%2FPipeInsertNodeTabletInsertionEvent.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9ldmVudC9jb21tb24vdGFibGV0L1BpcGVJbnNlcnROb2RlVGFibGV0SW5zZXJ0aW9uRXZlbnQuamF2YQ==) | 11.53% | [23 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../event/common/tsfile/PipeTsFileInsertionEvent.java](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fevent%2Fcommon%2Ftsfile%2FPipeTsFileInsertionEvent.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9ldmVudC9jb21tb24vdHNmaWxlL1BpcGVUc0ZpbGVJbnNlcnRpb25FdmVudC5qYXZh) | 15.38% | [11 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...gion/realtime/PipeRealtimeDataRegionExtractor.java](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fextractor%2Fdataregion%2Frealtime%2FPipeRealtimeDataRegionExtractor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9leHRyYWN0b3IvZGF0YXJlZ2lvbi9yZWFsdGltZS9QaXBlUmVhbHRpbWVEYXRhUmVnaW9uRXh0cmFjdG9yLmphdmE=) | 77.27% | [5 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ent/common/tablet/PipeRawTabletInsertionEvent.java](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fevent%2Fcommon%2Ftablet%2FPipeRawTabletInsertionEvent.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9ldmVudC9jb21tb24vdGFibGV0L1BpcGVSYXdUYWJsZXRJbnNlcnRpb25FdmVudC5qYXZh) | 0.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...rical/PipeHistoricalDataRegionTsFileExtractor.java](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fextractor%2Fdataregion%2Fhistorical%2FPipeHistoricalDataRegionTsFileExtractor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9leHRyYWN0b3IvZGF0YXJlZ2lvbi9oaXN0b3JpY2FsL1BpcGVIaXN0b3JpY2FsRGF0YVJlZ2lvblRzRmlsZUV4dHJhY3Rvci5qYXZh) | 33.33% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../iotdb/db/pipe/event/UserDefinedEnrichedEvent.java](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fevent%2FUserDefinedEnrichedEvent.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9ldmVudC9Vc2VyRGVmaW5lZEVucmljaGVkRXZlbnQuamF2YQ==) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ipe/event/common/heartbeat/PipeHeartbeatEvent.java](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fevent%2Fcommon%2Fheartbeat%2FPipeHeartbeatEvent.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9ldmVudC9jb21tb24vaGVhcnRiZWF0L1BpcGVIZWFydGJlYXRFdmVudC5qYXZh) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ipe/event/common/terminate/PipeTerminateEvent.java](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fevent%2Fcommon%2Fterminate%2FPipeTerminateEvent.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9ldmVudC9jb21tb24vdGVybWluYXRlL1BpcGVUZXJtaW5hdGVFdmVudC5qYXZh) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...otdb/db/pipe/event/realtime/PipeRealtimeEvent.java](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fevent%2Frealtime%2FPipeRealtimeEvent.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9ldmVudC9yZWFsdGltZS9QaXBlUmVhbHRpbWVFdmVudC5qYXZh) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...he/iotdb/commons/pipe/event/PipeSnapshotEvent.java](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&filepath=iotdb-core%2Fnode-commons%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fcommons%2Fpipe%2Fevent%2FPipeSnapshotEvent.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9ub2RlLWNvbW1vbnMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2lvdGRiL2NvbW1vbnMvcGlwZS9ldmVudC9QaXBlU25hcHNob3RFdmVudC5qYXZh) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| ... and [2 more](https://app.codecov.io/gh/apache/iotdb/pull/12751?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #12751 +/- ##
=========================================
Coverage 41.75% 41.75%
Complexity 71 71
=========================================
Files 3293 3293
Lines 207575 207632 +57
Branches 24666 24679 +13
=========================================
+ Hits 86670 86699 +29
- Misses 120905 120933 +28
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/iotdb/pull/12751?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "VGalaxies (via GitHub)" <gi...@apache.org>.
VGalaxies commented on code in PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#discussion_r1642742514
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tablet/PipeInsertNodeTabletInsertionEvent.java:
##########
@@ -205,29 +205,36 @@ public boolean isGeneratedByPipe() {
@Override
public boolean mayEventTimeOverlappedWithTimeRange() {
try {
- final InsertNode insertNode = getInsertNode();
+ final InsertNode insertNode = getInsertNodeViaCacheIfPossible();
+ if (Objects.isNull(insertNode)) {
+ return true;
+ }
+
if (insertNode instanceof InsertRowNode) {
final long timestamp = ((InsertRowNode) insertNode).getTime();
return startTime <= timestamp && timestamp <= endTime;
- } else if (insertNode instanceof InsertTabletNode) {
+ }
+
+ if (insertNode instanceof InsertTabletNode) {
final long[] timestamps = ((InsertTabletNode) insertNode).getTimes();
if (Objects.isNull(timestamps) || timestamps.length == 0) {
return false;
}
// We assume that `timestamps` is ordered.
return startTime <= timestamps[timestamps.length - 1] && timestamps[0] <= endTime;
- } else if (insertNode instanceof InsertRowsNode) {
+ }
+
+ if (insertNode instanceof InsertRowsNode) {
for (final InsertRowNode node : ((InsertRowsNode) insertNode).getInsertRowNodeList()) {
final long timestamp = node.getTime();
Review Comment:
use stream like [`5f79e07` (#12751)](https://github.com/apache/iotdb/pull/12751/commits/5f79e07b1ea0c2b5f8c713d229a36b5c2fbcc403#diff-34659d7ee3257a64a7863eb5b7aa3e6bf86f2f8ad9abaa9f5d8894108d70213bR265)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "SteveYurongSu (via GitHub)" <gi...@apache.org>.
SteveYurongSu merged PR #12751:
URL: https://github.com/apache/iotdb/pull/12751
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#issuecomment-2173433143
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=12751&metric=new_coverage&view=list) (required ≥ 80%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#issuecomment-2175256753
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=12751&metric=new_coverage&view=list) (required ≥ 80%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#issuecomment-2173083943
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=12751&metric=new_coverage&view=list) (required ≥ 80%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "VGalaxies (via GitHub)" <gi...@apache.org>.
VGalaxies commented on code in PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#discussion_r1642681203
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/extractor/dataregion/realtime/PipeRealtimeDataRegionExtractor.java:
##########
@@ -142,10 +147,33 @@ public void validate(final PipeParameterValidator validator) throws Exception {
EXTRACTOR_END_TIME_KEY,
realtimeDataExtractionEndTime));
}
+ } catch (final PipeParameterNotValidException e) {
+ throw e;
} catch (final Exception e) {
// compatible with the current validation framework
throw new PipeParameterNotValidException(e.getMessage());
}
+
+ final Set<String> sloppyOptionSet =
+ Arrays.stream(
+ parameters
+ .getStringOrDefault(
+ Arrays.asList(
+ EXTRACTOR_REALTIME_LOOSE_RANGE_KEY, SOURCE_REALTIME_LOOSE_RANGE_KEY),
+ EXTRACTOR_REALTIME_LOOSE_RANGE_DEFAULT_VALUE)
+ .split(","))
+ .map(String::trim)
+ .map(String::toLowerCase)
+ .collect(Collectors.toSet());
+ // Avoid empty string
+ sloppyOptionSet.remove("");
+ sloppyTimeRange = sloppyOptionSet.contains(EXTRACTOR_REALTIME_LOOSE_RANGE_TIME_VALUE);
+ sloppyPattern = sloppyOptionSet.contains(EXTRACTOR_REALTIME_LOOSE_RANGE_PATH_VALUE);
+ if (!sloppyOptionSet.isEmpty()) {
Review Comment:
remove "time" and "path" before checking?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#issuecomment-2174980924
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=12751&metric=new_coverage&view=list) (required ≥ 80%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: Support "source.realtime.loose-range" = "path" in iotdb-source [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #12751:
URL: https://github.com/apache/iotdb/pull/12751#issuecomment-2173276686
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751) **Quality Gate failed**
Failed conditions
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=12751&metric=new_coverage&view=list) (required ≥ 80%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12751)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org