You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/12/29 01:21:48 UTC
[GitHub] [dolphinscheduler] Radeity opened a new pull request, #13298: [Improvement][Master] Validate same content of input file when using task cache
Radeity opened a new pull request, #13298:
URL: https://github.com/apache/dolphinscheduler/pull/13298
<!--Thanks very much for contributing to Apache DolphinScheduler. Please review https://dolphinscheduler.apache.org/en-us/community/development/pull-request.html before opening a pull request.-->
## Purpose of the pull request
- This PR closes https://github.com/apache/dolphinscheduler/pull/13274.
<!--(For example: This pull request adds checkstyle plugin).-->
## Brief change log
- Add function `getFileChecksum` in `FileUtils` available for both file and directory. When calculate checksum for a directory, it will calculate all files checksum under the directory and concatenate them together, the order is provided by file-system interface.
- Upload checksum file (with suffix `.crc`) together with output file when uploading for task file transfer.
- Download checksum file and replace the value of file property with content of checksum file.
<!--*(for example:)*
- *Add maven-checkstyle-plugin to root pom.xml*
-->
## Verify this pull request
- Already add some UT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-code-scanning[bot] commented on a diff in pull request #13298: [Improvement][Master] Validate same content of input file when using task cache
Posted by GitBox <gi...@apache.org>.
github-code-scanning[bot] commented on code in PR #13298:
URL: https://github.com/apache/dolphinscheduler/pull/13298#discussion_r1058700526
##########
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/FileUtils.java:
##########
@@ -264,4 +267,41 @@
}
}
+ /**
+ * Calculate file checksum with CRC32 algorithm
+ * @param pathName
+ * @return checksum of file/dir
+ */
+ public static String getFileChecksum(String pathName) throws IOException {
+ CRC32 crc32 = new CRC32();
+ File file = new File(pathName);
+ String crcString = "";
+ if (file.isDirectory()) {
+ // file system interface remains the same order
+ String[] subPaths = file.list();
+ StringBuilder concatenatedCRC = new StringBuilder();
+ for (String subPath : subPaths) {
+ concatenatedCRC.append(getFileChecksum(pathName + FOLDER_SEPARATOR + subPath));
+ }
+ crcString = concatenatedCRC.toString();
+ } else {
+ FileInputStream fileInputStream = null;
+ CheckedInputStream checkedInputStream = null;
+ try {
+ fileInputStream = new FileInputStream(pathName);
+ checkedInputStream = new CheckedInputStream(fileInputStream, crc32);
+ while (checkedInputStream.read() != -1) {
+ }
+ } catch (IOException e) {
+ throw new IOException("Calculate checksum error.");
+ } finally {
+ fileInputStream.close();
Review Comment:
## Dereferenced variable may be null
Variable [fileInputStream](1) may be null at this access because of [this](2) assignment.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/2457)
##########
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/FileUtils.java:
##########
@@ -264,4 +267,41 @@
}
}
+ /**
+ * Calculate file checksum with CRC32 algorithm
+ * @param pathName
+ * @return checksum of file/dir
+ */
+ public static String getFileChecksum(String pathName) throws IOException {
+ CRC32 crc32 = new CRC32();
+ File file = new File(pathName);
+ String crcString = "";
+ if (file.isDirectory()) {
+ // file system interface remains the same order
+ String[] subPaths = file.list();
+ StringBuilder concatenatedCRC = new StringBuilder();
+ for (String subPath : subPaths) {
+ concatenatedCRC.append(getFileChecksum(pathName + FOLDER_SEPARATOR + subPath));
+ }
+ crcString = concatenatedCRC.toString();
+ } else {
+ FileInputStream fileInputStream = null;
+ CheckedInputStream checkedInputStream = null;
+ try {
+ fileInputStream = new FileInputStream(pathName);
+ checkedInputStream = new CheckedInputStream(fileInputStream, crc32);
+ while (checkedInputStream.read() != -1) {
+ }
+ } catch (IOException e) {
+ throw new IOException("Calculate checksum error.");
+ } finally {
+ fileInputStream.close();
+ checkedInputStream.close();
Review Comment:
## Dereferenced variable may be null
Variable [checkedInputStream](1) may be null at this access because of [this](2) assignment.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/2458)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13298: [Improvement][Master] Validate same content of input file when using task cache
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #13298:
URL: https://github.com/apache/dolphinscheduler/pull/13298#issuecomment-1367722564
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13298)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list) [83.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13298: [Improvement][Master] Validate same content of input file when using task cache
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #13298:
URL: https://github.com/apache/dolphinscheduler/pull/13298#issuecomment-1367038336
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13298)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [2 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list) [84.6% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-code-scanning[bot] commented on a diff in pull request #13298: [Improvement][Master] Validate same content of input file when using task cache
Posted by GitBox <gi...@apache.org>.
github-code-scanning[bot] commented on code in PR #13298:
URL: https://github.com/apache/dolphinscheduler/pull/13298#discussion_r1058685086
##########
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/FileUtils.java:
##########
@@ -264,4 +267,38 @@
}
}
+ /**
+ * Calculate file checksum with CRC32 algorithm
+ *
+ * @param pathName
+ * @return
+ * @throws IOException
+ */
+ public static String getFileChecksum(String pathName) {
+ CRC32 crc32 = new CRC32();
+ File file = new File(pathName);
+ String crcString = "";
+ if (file.isDirectory()) {
+ // file system interface remains the same order
+ String[] subPaths = file.list();
+ for (String subPath : subPaths) {
+ crcString += getFileChecksum(pathName + FOLDER_SEPARATOR + subPath);
+ }
+ } else {
+ FileInputStream fileInputStream = null;
+ try {
+ fileInputStream = new FileInputStream(pathName);
Review Comment:
## Potential input resource leak
This FileInputStream is not always closed on method exit.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/2455)
##########
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/FileUtils.java:
##########
@@ -264,4 +267,38 @@
}
}
+ /**
+ * Calculate file checksum with CRC32 algorithm
+ *
+ * @param pathName
+ * @return
+ * @throws IOException
Review Comment:
## Javadoc has impossible 'throws' tag
Javadoc for getFileChecksum claims to throw IOException but this is impossible.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/2454)
##########
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/utils/TaskCacheUtils.java:
##########
@@ -139,9 +161,40 @@
.filter(property -> propertyInSet.contains(property.getProp()))
.sorted(Comparator.comparing(Property::getProp))
.collect(Collectors.toList());
+
+ varPool.forEach(property -> {
+ if (property.getType() == DataType.FILE) {
+ property.setValue(fileCheckSumMap.get(property.getValue()));
+ }
+ });
return JSONUtils.toJsonString(varPool);
}
+ /**
+ * get checksum from crc32 file of file property in varPool
+ * cache can be used if content of upstream output files are the same
+ * @param fileProperty
+ * @param context
+ * @param storageOperate
+ */
+ public static String getValCheckSum(Property fileProperty, TaskExecutionContext context,
+ StorageOperate storageOperate) {
+ String resourceCRCPath = fileProperty.getValue() + CRC_SUFFIX;
+ String resourceCRCWholePath = storageOperate.getResourceFileName(context.getTenantCode(), resourceCRCPath);
+ String targetPath = String.format("%s/%s", context.getExecutePath(), resourceCRCPath);
+ logger.info("{} --- Remote:{} to Local:{}", "CRC file", resourceCRCWholePath, targetPath);
+ String crcString = "";
+ try {
+ storageOperate.download(context.getTenantCode(), resourceCRCWholePath, targetPath, false,
+ true);
+ crcString = FileUtils.readFile2Str(new FileInputStream(targetPath));
Review Comment:
## Potential input resource leak
This FileInputStream is not always closed on method exit.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/2456)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13298: [Improvement][Master] Validate same content of input file when using task cache
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #13298:
URL: https://github.com/apache/dolphinscheduler/pull/13298#issuecomment-1367734498
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13298)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list) [83.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #13298: [Improvement][Master] Validate same content of input file when using task cache
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #13298:
URL: https://github.com/apache/dolphinscheduler/pull/13298#issuecomment-1367018198
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/13298?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#13298](https://codecov.io/gh/apache/dolphinscheduler/pull/13298?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f6a0a97) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/8870464c02ae2bf131c52e016e2a1af8d2196f27?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8870464) will **increase** coverage by `0.06%`.
> The diff coverage is `82.35%`.
```diff
@@ Coverage Diff @@
## dev #13298 +/- ##
============================================
+ Coverage 39.53% 39.59% +0.06%
- Complexity 4293 4301 +8
============================================
Files 1072 1072
Lines 40558 40581 +23
Branches 4670 4663 -7
============================================
+ Hits 16035 16069 +34
+ Misses 22741 22727 -14
- Partials 1782 1785 +3
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/13298?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...e/dolphinscheduler/common/constants/Constants.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13298/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2NvbnN0YW50cy9Db25zdGFudHMuamF2YQ==) | `75.00% <ø> (ø)` | |
| [...ver/master/consumer/TaskPriorityQueueConsumer.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13298/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1tYXN0ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9jb25zdW1lci9UYXNrUHJpb3JpdHlRdWV1ZUNvbnN1bWVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...che/dolphinscheduler/dao/utils/TaskCacheUtils.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13298/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYW8vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvZGFvL3V0aWxzL1Rhc2tDYWNoZVV0aWxzLmphdmE=) | `86.76% <78.26%> (-4.54%)` | :arrow_down: |
| [...pache/dolphinscheduler/common/utils/FileUtils.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13298/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3V0aWxzL0ZpbGVVdGlscy5qYXZh) | `60.22% <83.33%> (+5.94%)` | :arrow_up: |
| [...er/server/worker/utils/TaskFilesTransferUtils.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13298/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci13b3JrZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci91dGlscy9UYXNrRmlsZXNUcmFuc2ZlclV0aWxzLmphdmE=) | `80.37% <100.00%> (+1.37%)` | :arrow_up: |
| [...erver/master/processor/queue/TaskEventService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13298/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1tYXN0ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvcXVldWUvVGFza0V2ZW50U2VydmljZS5qYXZh) | `75.00% <0.00%> (-5.36%)` | :arrow_down: |
| [...uler/api/controller/ProcessInstanceController.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13298/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL2NvbnRyb2xsZXIvUHJvY2Vzc0luc3RhbmNlQ29udHJvbGxlci5qYXZh) | `68.18% <0.00%> (-2.04%)` | :arrow_down: |
| [...java/org/apache/dolphinscheduler/dao/AlertDao.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13298/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYW8vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvZGFvL0FsZXJ0RGFvLmphdmE=) | `28.46% <0.00%> (-0.86%)` | :arrow_down: |
| [...pache/dolphinscheduler/common/utils/DateUtils.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13298/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3V0aWxzL0RhdGVVdGlscy5qYXZh) | `68.30% <0.00%> (-0.55%)` | :arrow_down: |
| [...inscheduler/service/alert/ProcessAlertManager.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13298/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3NlcnZpY2UvYWxlcnQvUHJvY2Vzc0FsZXJ0TWFuYWdlci5qYXZh) | `41.70% <0.00%> (-0.38%)` | :arrow_down: |
| ... and [9 more](https://codecov.io/gh/apache/dolphinscheduler/pull/13298/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13298: [Improvement][Master] Validate same content of input file when using task cache
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #13298:
URL: https://github.com/apache/dolphinscheduler/pull/13298#issuecomment-1367019514
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13298)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list) [84.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13298: [Improvement][Master] Validate same content of input file when using task cache
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #13298:
URL: https://github.com/apache/dolphinscheduler/pull/13298#issuecomment-1367721029
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13298)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list) [84.1% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] jieguangzhou merged pull request #13298: [Improvement][Master] Validate same content of input file when using task cache
Posted by GitBox <gi...@apache.org>.
jieguangzhou merged PR #13298:
URL: https://github.com/apache/dolphinscheduler/pull/13298
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13298: [Improvement][Master] Validate same content of input file when using task cache
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #13298:
URL: https://github.com/apache/dolphinscheduler/pull/13298#issuecomment-1367020189
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13298)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list) [84.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13298: [Improvement][Master] Validate same content of input file when using task cache
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #13298:
URL: https://github.com/apache/dolphinscheduler/pull/13298#issuecomment-1367038146
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13298)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [2 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list) [84.6% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13298: [Improvement][Master] Validate same content of input file when using task cache
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #13298:
URL: https://github.com/apache/dolphinscheduler/pull/13298#issuecomment-1367733995
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13298)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13298&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list) [84.1% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13298&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org