You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "NSAmelchev (via GitHub)" <gi...@apache.org> on 2023/06/06 09:46:18 UTC
[GitHub] [ignite] NSAmelchev opened a new pull request, #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
NSAmelchev opened a new pull request, #10766:
URL: https://github.com/apache/ignite/pull/10766
…led.
Thank you for submitting the pull request to the Apache Ignite.
In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:
### The Contribution Checklist
- [ ] There is a single JIRA ticket related to the pull request.
- [ ] The web-link to the pull request is attached to the JIRA ticket.
- [ ] The JIRA ticket has the _Patch Available_ state.
- [ ] The pull request body describes changes that have been made.
The description explains _WHAT_ and _WHY_ was made instead of _HOW_.
- [ ] The pull request title is treated as the final commit message.
The following pattern must be used: `IGNITE-XXXX Change summary` where `XXXX` - number of JIRA issue.
- [ ] A reviewer has been mentioned through the JIRA comments
(see [the Maintainers list](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers))
- [ ] The pull request has been checked by the Teamcity Bot and
the `green visa` attached to the JIRA ticket (see [TC.Bot: Check PR](https://mtcga.gridgain.com/prs.html))
### Notes
- [How to Contribute](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute)
- [Coding abbreviation rules](https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules)
- [Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines)
- [Apache Ignite Teamcity Bot](https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot)
If you need any help, please email dev@ignite.apache.org or ask anу advice on http://asf.slack.com _#ignite_ channel.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1692314935
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 The version of Java (11.0.20) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1654475977
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] nizhikov commented on a diff in pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "nizhikov (via GitHub)" <gi...@apache.org>.
nizhikov commented on code in PR #10766:
URL: https://github.com/apache/ignite/pull/10766#discussion_r1256025145
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/FileWriteAheadLogManager.java:
##########
@@ -1818,6 +1837,37 @@ public long maxWalSegmentSize() {
return maxWalSegmentSize;
}
+ /** @return {@code True} if log record should be skipped when CDC is disabled. */
+ private boolean skipIfCdcDisabled(WALRecord rec) {
+ if (!inMemoryCdc || rec instanceof CdcDisableRecord || !cdcDisabled.getOrDefault(false))
+ return false;
+
+ LT.warn(log, "Logging CDC data records to WAL skipped. The '" + CDC_DISABLED +
Review Comment:
LT.warn effectively creates an object to check timeout of logging on every invocation
```
IgniteBiTuple<Class<? extends Throwable>, String> tup =
e != null && !byMsg ? F.<Class<? extends Throwable>, String>t(e.getClass(), e.getMessage()) :
F.<Class<? extends Throwable>, String>t(null, longMsg);
```
So we shouldn't call it for every DataRecord that will be logged on every entry change.
Let's log changes of cdcDisabled state only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1613814362
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [7 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] nizhikov commented on a diff in pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "nizhikov (via GitHub)" <gi...@apache.org>.
nizhikov commented on code in PR #10766:
URL: https://github.com/apache/ignite/pull/10766#discussion_r1255983617
##########
modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java:
##########
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool
try {
lock.tryLock(1);
- try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) {
- Set<File> delete = new HashSet<>();
+ Long lostSgmnt = findLastLostSegement();
- AtomicLong lastSgmnt = new AtomicLong(-1);
+ if (lostSgmnt != null)
+ deleteAll(lostSgmnt);
- cdcFiles
- .filter(p -> WAL_SEGMENT_FILE_FILTER.accept(p.toFile()))
- .sorted(Comparator.comparingLong(FileWriteAheadLogManager::segmentIndex)
- .reversed()) // Sort by segment index.
- .forEach(path -> {
- long idx = FileWriteAheadLogManager.segmentIndex(path);
+ Long cdcDisableSgmnt = findLastCdcDisableSegment();
- if (lastSgmnt.get() == -1 || lastSgmnt.get() - idx == 1) {
- lastSgmnt.set(idx);
+ if (cdcDisableSgmnt != null)
+ deleteAll(cdcDisableSgmnt);
- return;
- }
+ if (lostSgmnt != null || cdcDisableSgmnt != null)
+ resetWalState();
+ else {
Review Comment:
brace can be omitted
```
else if (log.isDebugEnabled)
log.debug("Here!");
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] nizhikov commented on a diff in pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "nizhikov (via GitHub)" <gi...@apache.org>.
nizhikov commented on code in PR #10766:
URL: https://github.com/apache/ignite/pull/10766#discussion_r1255987713
##########
modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java:
##########
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool
try {
lock.tryLock(1);
- try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) {
- Set<File> delete = new HashSet<>();
+ Long lostSgmnt = findLastLostSegement();
- AtomicLong lastSgmnt = new AtomicLong(-1);
+ if (lostSgmnt != null)
+ deleteAll(lostSgmnt);
- cdcFiles
- .filter(p -> WAL_SEGMENT_FILE_FILTER.accept(p.toFile()))
- .sorted(Comparator.comparingLong(FileWriteAheadLogManager::segmentIndex)
- .reversed()) // Sort by segment index.
- .forEach(path -> {
- long idx = FileWriteAheadLogManager.segmentIndex(path);
+ Long cdcDisableSgmnt = findLastCdcDisableSegment();
- if (lastSgmnt.get() == -1 || lastSgmnt.get() - idx == 1) {
- lastSgmnt.set(idx);
+ if (cdcDisableSgmnt != null)
+ deleteAll(cdcDisableSgmnt);
- return;
- }
+ if (lostSgmnt != null || cdcDisableSgmnt != null)
+ resetWalState();
+ else {
+ if (log.isInfoEnabled())
+ log.info("Lost segment CDC links or CDC disable record were not found.");
+ }
+ }
+ catch (IgniteCheckedException e) {
+ throw new IgniteException("Failed to delete lost segment CDC links. " +
+ "Unable to acquire lock to lock CDC folder. Make sure a CDC app is shut down " +
+ "[dir=" + walCdcDir.getAbsolutePath() + ", reason=" + e.getMessage() + ']');
+ }
+ finally {
+ U.closeQuiet(lock);
+ }
+
+ return null;
+ }
- delete.add(path.toFile());
- });
+ /** @return The index of the segment previous to the last gap or {@code null} if no gaps were found. */
+ private Long findLastLostSegement() {
+ AtomicReference<Long> lastLostSgmnt = new AtomicReference<>();
+ AtomicLong lastSgmnt = new AtomicLong(-1);
- if (delete.isEmpty()) {
- log.info("Lost segment CDC links were not found.");
+ consumeCdcSegments(segment -> {
+ if (lastLostSgmnt.get() != null)
+ return;
- return null;
- }
+ long idx = FileWriteAheadLogManager.segmentIndex(segment);
- log.info("Found lost segment CDC links. The following links will be deleted: " + delete);
+ if (lastSgmnt.get() == -1 || lastSgmnt.get() - idx == 1) {
+ lastSgmnt.set(idx);
+
+ return;
+ }
+
+ if (log.isInfoEnabled())
+ log.info("Found lost segment CDC links [lastLostSgmntIdx=" + idx + ']');
+
+ lastLostSgmnt.set(idx);
+ });
+
+ return lastLostSgmnt.get();
+ }
- delete.forEach(file -> {
- if (!file.delete()) {
- throw new IgniteException("Failed to delete lost segment CDC link [file=" +
- file.getAbsolutePath() + ']');
- }
+ /** @return The index of the segment that contains the last {@link CdcDisableRecord}. */
+ private Long findLastCdcDisableSegment() {
+ AtomicReference<Long> lastRec = new AtomicReference<>();
- log.info("Segment CDC link deleted [file=" + file.getAbsolutePath() + ']');
- });
+ consumeCdcSegments(segment -> {
+ if (lastRec.get() != null)
+ return;
- Path stateDir = walCdcDir.toPath().resolve(STATE_DIR);
+ if (log.isInfoEnabled())
+ log.info("Processing CDC segment [segment=" + segment + ']');
- if (stateDir.toFile().exists()) {
- File walState = stateDir.resolve(WAL_STATE_FILE_NAME).toFile();
+ IgniteWalIteratorFactory.IteratorParametersBuilder builder =
+ new IgniteWalIteratorFactory.IteratorParametersBuilder()
+ .log(log)
+ .sharedContext(ignite.context().cache().context())
+ .filesOrDirs(segment.toFile())
+ .addFilter((type, ptr) -> type == CDC_DISABLE);
- if (walState.exists() && !walState.delete()) {
- throw new IgniteException("Failed to delete wal state file [file=" +
- walState.getAbsolutePath() + ']');
- }
+ if (ignite.configuration().getDataStorageConfiguration().getPageSize() != 0)
+ builder.pageSize(ignite.configuration().getDataStorageConfiguration().getPageSize());
+
+ try (WALIterator it = new IgniteWalIteratorFactory(log).iterator(builder)) {
+ if (it.hasNext()) {
+ if (log.isInfoEnabled())
+ log.info("Found CDC disable record [ptr=" + it.next().get1() + ']');
+
+ lastRec.set(FileWriteAheadLogManager.segmentIndex(segment));
}
}
- catch (IOException e) {
- throw new RuntimeException("Failed to delete lost segment CDC links.", e);
+ catch (IgniteCheckedException e) {
+ throw new IgniteException("Failed to read CDC segment [path=" + segment + ']', e);
}
+ });
+
+ return lastRec.get();
+ }
+
+ /** */
+ private void resetWalState() {
Review Comment:
deleteCdcWalState
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] nizhikov commented on a diff in pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "nizhikov (via GitHub)" <gi...@apache.org>.
nizhikov commented on code in PR #10766:
URL: https://github.com/apache/ignite/pull/10766#discussion_r1256025145
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/FileWriteAheadLogManager.java:
##########
@@ -1818,6 +1837,37 @@ public long maxWalSegmentSize() {
return maxWalSegmentSize;
}
+ /** @return {@code True} if log record should be skipped when CDC is disabled. */
+ private boolean skipIfCdcDisabled(WALRecord rec) {
+ if (!inMemoryCdc || rec instanceof CdcDisableRecord || !cdcDisabled.getOrDefault(false))
+ return false;
+
+ LT.warn(log, "Logging CDC data records to WAL skipped. The '" + CDC_DISABLED +
Review Comment:
LT.warn effectively creates an object to check timeout of logging
```
IgniteBiTuple<Class<? extends Throwable>, String> tup =
e != null && !byMsg ? F.<Class<? extends Throwable>, String>t(e.getClass(), e.getMessage()) :
F.<Class<? extends Throwable>, String>t(null, longMsg);
```
So we shouldn't call it for every DataRecord that will be logged on every entry change.
Let's log changes of cdcDisabled state only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1624384090
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [8 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1613205104
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [7 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1701702381
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 The version of Java (11.0.20) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] nizhikov commented on a diff in pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "nizhikov (via GitHub)" <gi...@apache.org>.
nizhikov commented on code in PR #10766:
URL: https://github.com/apache/ignite/pull/10766#discussion_r1255997863
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/FileWriteAheadLogManager.java:
##########
@@ -515,8 +521,18 @@ public void setFileIOFactory(FileIOFactory ioFactory) {
name, oldVal, newVal));
}
- if (newVal != null && newVal)
+ if (newVal != null && newVal) {
Review Comment:
A few linew below you check both values - `oldVal && !newVal`.
Here you check only `newVal`.
Does it mean that disable record can be recorded twice? When oldVal == newVal
So, it seems you should keep logic of both check consistent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1613364771
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [7 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1620187634
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [6 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1706390482
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 The version of Java (11.0.20) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1625589678
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [8 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] nizhikov commented on a diff in pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "nizhikov (via GitHub)" <gi...@apache.org>.
nizhikov commented on code in PR #10766:
URL: https://github.com/apache/ignite/pull/10766#discussion_r1255964521
##########
modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java:
##########
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool
try {
lock.tryLock(1);
- try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) {
- Set<File> delete = new HashSet<>();
+ Long lostSgmnt = findLastLostSegement();
- AtomicLong lastSgmnt = new AtomicLong(-1);
+ if (lostSgmnt != null)
+ deleteAll(lostSgmnt);
- cdcFiles
- .filter(p -> WAL_SEGMENT_FILE_FILTER.accept(p.toFile()))
- .sorted(Comparator.comparingLong(FileWriteAheadLogManager::segmentIndex)
- .reversed()) // Sort by segment index.
- .forEach(path -> {
- long idx = FileWriteAheadLogManager.segmentIndex(path);
+ Long cdcDisableSgmnt = findLastCdcDisableSegment();
Review Comment:
findLastCdcDisableSegment -> findLastSegmentWithCdcDisabledRecord.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1593159311
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [4 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1592664626
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [4 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] NSAmelchev commented on a diff in pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "NSAmelchev (via GitHub)" <gi...@apache.org>.
NSAmelchev commented on code in PR #10766:
URL: https://github.com/apache/ignite/pull/10766#discussion_r1256004639
##########
modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java:
##########
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool
try {
lock.tryLock(1);
- try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) {
- Set<File> delete = new HashSet<>();
+ Long lostSgmnt = findLastLostSegement();
- AtomicLong lastSgmnt = new AtomicLong(-1);
+ if (lostSgmnt != null)
+ deleteAll(lostSgmnt);
- cdcFiles
- .filter(p -> WAL_SEGMENT_FILE_FILTER.accept(p.toFile()))
- .sorted(Comparator.comparingLong(FileWriteAheadLogManager::segmentIndex)
- .reversed()) // Sort by segment index.
- .forEach(path -> {
- long idx = FileWriteAheadLogManager.segmentIndex(path);
+ Long cdcDisableSgmnt = findLastCdcDisableSegment();
- if (lastSgmnt.get() == -1 || lastSgmnt.get() - idx == 1) {
- lastSgmnt.set(idx);
+ if (cdcDisableSgmnt != null)
+ deleteAll(cdcDisableSgmnt);
- return;
- }
+ if (lostSgmnt != null || cdcDisableSgmnt != null)
+ resetWalState();
+ else {
Review Comment:
Fixed
##########
modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java:
##########
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool
try {
lock.tryLock(1);
- try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) {
- Set<File> delete = new HashSet<>();
+ Long lostSgmnt = findLastLostSegement();
- AtomicLong lastSgmnt = new AtomicLong(-1);
+ if (lostSgmnt != null)
+ deleteAll(lostSgmnt);
- cdcFiles
- .filter(p -> WAL_SEGMENT_FILE_FILTER.accept(p.toFile()))
- .sorted(Comparator.comparingLong(FileWriteAheadLogManager::segmentIndex)
- .reversed()) // Sort by segment index.
- .forEach(path -> {
- long idx = FileWriteAheadLogManager.segmentIndex(path);
+ Long cdcDisableSgmnt = findLastCdcDisableSegment();
- if (lastSgmnt.get() == -1 || lastSgmnt.get() - idx == 1) {
- lastSgmnt.set(idx);
+ if (cdcDisableSgmnt != null)
+ deleteAll(cdcDisableSgmnt);
- return;
- }
+ if (lostSgmnt != null || cdcDisableSgmnt != null)
+ resetWalState();
+ else {
+ if (log.isInfoEnabled())
+ log.info("Lost segment CDC links or CDC disable record were not found.");
+ }
+ }
+ catch (IgniteCheckedException e) {
+ throw new IgniteException("Failed to delete lost segment CDC links. " +
+ "Unable to acquire lock to lock CDC folder. Make sure a CDC app is shut down " +
+ "[dir=" + walCdcDir.getAbsolutePath() + ", reason=" + e.getMessage() + ']');
+ }
+ finally {
+ U.closeQuiet(lock);
+ }
+
+ return null;
+ }
- delete.add(path.toFile());
- });
+ /** @return The index of the segment previous to the last gap or {@code null} if no gaps were found. */
+ private Long findLastLostSegement() {
+ AtomicReference<Long> lastLostSgmnt = new AtomicReference<>();
+ AtomicLong lastSgmnt = new AtomicLong(-1);
- if (delete.isEmpty()) {
- log.info("Lost segment CDC links were not found.");
+ consumeCdcSegments(segment -> {
+ if (lastLostSgmnt.get() != null)
+ return;
- return null;
- }
+ long idx = FileWriteAheadLogManager.segmentIndex(segment);
- log.info("Found lost segment CDC links. The following links will be deleted: " + delete);
+ if (lastSgmnt.get() == -1 || lastSgmnt.get() - idx == 1) {
+ lastSgmnt.set(idx);
+
+ return;
+ }
+
+ if (log.isInfoEnabled())
+ log.info("Found lost segment CDC links [lastLostSgmntIdx=" + idx + ']');
+
+ lastLostSgmnt.set(idx);
+ });
+
+ return lastLostSgmnt.get();
+ }
- delete.forEach(file -> {
- if (!file.delete()) {
- throw new IgniteException("Failed to delete lost segment CDC link [file=" +
- file.getAbsolutePath() + ']');
- }
+ /** @return The index of the segment that contains the last {@link CdcDisableRecord}. */
+ private Long findLastCdcDisableSegment() {
+ AtomicReference<Long> lastRec = new AtomicReference<>();
- log.info("Segment CDC link deleted [file=" + file.getAbsolutePath() + ']');
- });
+ consumeCdcSegments(segment -> {
+ if (lastRec.get() != null)
+ return;
- Path stateDir = walCdcDir.toPath().resolve(STATE_DIR);
+ if (log.isInfoEnabled())
+ log.info("Processing CDC segment [segment=" + segment + ']');
- if (stateDir.toFile().exists()) {
- File walState = stateDir.resolve(WAL_STATE_FILE_NAME).toFile();
+ IgniteWalIteratorFactory.IteratorParametersBuilder builder =
+ new IgniteWalIteratorFactory.IteratorParametersBuilder()
+ .log(log)
+ .sharedContext(ignite.context().cache().context())
+ .filesOrDirs(segment.toFile())
+ .addFilter((type, ptr) -> type == CDC_DISABLE);
- if (walState.exists() && !walState.delete()) {
- throw new IgniteException("Failed to delete wal state file [file=" +
- walState.getAbsolutePath() + ']');
- }
+ if (ignite.configuration().getDataStorageConfiguration().getPageSize() != 0)
+ builder.pageSize(ignite.configuration().getDataStorageConfiguration().getPageSize());
+
+ try (WALIterator it = new IgniteWalIteratorFactory(log).iterator(builder)) {
+ if (it.hasNext()) {
+ if (log.isInfoEnabled())
+ log.info("Found CDC disable record [ptr=" + it.next().get1() + ']');
+
+ lastRec.set(FileWriteAheadLogManager.segmentIndex(segment));
}
}
- catch (IOException e) {
- throw new RuntimeException("Failed to delete lost segment CDC links.", e);
+ catch (IgniteCheckedException e) {
+ throw new IgniteException("Failed to read CDC segment [path=" + segment + ']', e);
}
+ });
+
+ return lastRec.get();
+ }
+
+ /** */
+ private void resetWalState() {
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] NSAmelchev commented on a diff in pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "NSAmelchev (via GitHub)" <gi...@apache.org>.
NSAmelchev commented on code in PR #10766:
URL: https://github.com/apache/ignite/pull/10766#discussion_r1256003572
##########
modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java:
##########
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool
try {
lock.tryLock(1);
- try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) {
- Set<File> delete = new HashSet<>();
+ Long lostSgmnt = findLastLostSegement();
Review Comment:
Fixed
##########
modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java:
##########
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool
try {
lock.tryLock(1);
- try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) {
- Set<File> delete = new HashSet<>();
+ Long lostSgmnt = findLastLostSegement();
- AtomicLong lastSgmnt = new AtomicLong(-1);
+ if (lostSgmnt != null)
+ deleteAll(lostSgmnt);
Review Comment:
Fixed
##########
modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java:
##########
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool
try {
lock.tryLock(1);
- try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) {
- Set<File> delete = new HashSet<>();
+ Long lostSgmnt = findLastLostSegement();
- AtomicLong lastSgmnt = new AtomicLong(-1);
+ if (lostSgmnt != null)
+ deleteAll(lostSgmnt);
- cdcFiles
- .filter(p -> WAL_SEGMENT_FILE_FILTER.accept(p.toFile()))
- .sorted(Comparator.comparingLong(FileWriteAheadLogManager::segmentIndex)
- .reversed()) // Sort by segment index.
- .forEach(path -> {
- long idx = FileWriteAheadLogManager.segmentIndex(path);
+ Long cdcDisableSgmnt = findLastCdcDisableSegment();
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] nizhikov commented on a diff in pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "nizhikov (via GitHub)" <gi...@apache.org>.
nizhikov commented on code in PR #10766:
URL: https://github.com/apache/ignite/pull/10766#discussion_r1255959009
##########
modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java:
##########
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool
try {
lock.tryLock(1);
- try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) {
- Set<File> delete = new HashSet<>();
+ Long lostSgmnt = findLastLostSegement();
Review Comment:
lostSgmnt -> lastSegBeforeSkip
findLastLostSegement -> findLastSegementBeforeSkip
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1620013432
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [7 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1656698782
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] nizhikov commented on a diff in pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "nizhikov (via GitHub)" <gi...@apache.org>.
nizhikov commented on code in PR #10766:
URL: https://github.com/apache/ignite/pull/10766#discussion_r1255964521
##########
modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java:
##########
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool
try {
lock.tryLock(1);
- try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) {
- Set<File> delete = new HashSet<>();
+ Long lostSgmnt = findLastLostSegement();
- AtomicLong lastSgmnt = new AtomicLong(-1);
+ if (lostSgmnt != null)
+ deleteAll(lostSgmnt);
- cdcFiles
- .filter(p -> WAL_SEGMENT_FILE_FILTER.accept(p.toFile()))
- .sorted(Comparator.comparingLong(FileWriteAheadLogManager::segmentIndex)
- .reversed()) // Sort by segment index.
- .forEach(path -> {
- long idx = FileWriteAheadLogManager.segmentIndex(path);
+ Long cdcDisableSgmnt = findLastCdcDisableSegment();
Review Comment:
findLastCdcDisableSegment -> findLastSegmentWithCdcDisabledRecord.
cdcDisableSgmnt -> lastSegWithDisabledRec
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] nizhikov commented on a diff in pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "nizhikov (via GitHub)" <gi...@apache.org>.
nizhikov commented on code in PR #10766:
URL: https://github.com/apache/ignite/pull/10766#discussion_r1255963016
##########
modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java:
##########
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool
try {
lock.tryLock(1);
- try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) {
- Set<File> delete = new HashSet<>();
+ Long lostSgmnt = findLastLostSegement();
- AtomicLong lastSgmnt = new AtomicLong(-1);
+ if (lostSgmnt != null)
Review Comment:
if (seg != null) -> this check can be moved to the method.
It seem, that will simplify the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] nizhikov commented on a diff in pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "nizhikov (via GitHub)" <gi...@apache.org>.
nizhikov commented on code in PR #10766:
URL: https://github.com/apache/ignite/pull/10766#discussion_r1255961857
##########
modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java:
##########
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool
try {
lock.tryLock(1);
- try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) {
- Set<File> delete = new HashSet<>();
+ Long lostSgmnt = findLastLostSegement();
- AtomicLong lastSgmnt = new AtomicLong(-1);
+ if (lostSgmnt != null)
+ deleteAll(lostSgmnt);
Review Comment:
deleteAll -> deleteAllUntil or
deleteAllPrevious(lostSgmnt + 1);
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] nizhikov commented on a diff in pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "nizhikov (via GitHub)" <gi...@apache.org>.
nizhikov commented on code in PR #10766:
URL: https://github.com/apache/ignite/pull/10766#discussion_r1255959009
##########
modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java:
##########
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool
try {
lock.tryLock(1);
- try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) {
- Set<File> delete = new HashSet<>();
+ Long lostSgmnt = findLastLostSegement();
Review Comment:
lostSgmnt -> lastSegBeforeSkip
findLastLostSegement -> findLastSegmentBeforeSkip
Be aware about typo in segEment in method name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1624845053
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [8 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1578320705
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10766: IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10766:
URL: https://github.com/apache/ignite/pull/10766#issuecomment-1613120247
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10766)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10766&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10766&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10766&metric=new_duplicated_lines_density&view=list)
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=sonarcloud-welcome)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org