You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "Vladsz83 (via GitHub)" <gi...@apache.org> on 2026/10/08 11:18:19 UTC

[PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Vladsz83 opened a new pull request, #13666:
URL: https://github.com/apache/ignite/pull/13666

   Thank you for submitting the pull request to the Apache Ignite.
   
   In order to streamline the review of the contribution 
   we ask you to ensure the following steps have been taken:
   
   ### The Contribution Checklist
   - [ ] There is a single JIRA ticket related to the pull request. 
   - [ ] The web-link to the pull request is attached to the JIRA ticket.
   - [ ] The JIRA ticket has the _Patch Available_ state.
   - [ ] The pull request body describes changes that have been made. 
   The description explains _WHAT_ and _WHY_ was made instead of _HOW_.
   - [ ] The pull request title is treated as the final commit message. 
   The following pattern must be used: `IGNITE-XXXX Change summary` where `XXXX` - number of JIRA issue.
   - [ ] A reviewer has been mentioned through the JIRA comments 
   (see [the Maintainers list](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers)) 
   - [ ] The pull request has been checked by the Teamcity Bot and 
   the `green visa` attached to the JIRA ticket (see tab `PR Check` at [TC.Bot - Instance 1](https://tcbot2.sbt-ignite-dev.ru/prs.html) or [TC.Bot - Instance 2](https://mtcga.gridgain.com/prs.html))
   
   ### Notes
   - [How to Contribute](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute)
   - [Coding abbreviation rules](https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules)
   - [Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines)
   - [Apache Ignite Teamcity Bot](https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot)
   
   If you need any help, please email dev@ignite.apache.org or ask anу advice on http://asf.slack.com _#ignite_ channel.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4363487434


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");
+
+            printer.accept("\tNode '%s' [uuid=%s]:".formatted(res.consistentIds()[nodeIdx], res.nodesIds()[nodeIdx]));
+
+            SnapshotListJobResult nodeResult = res.nodesSnapshots()[nodeIdx];
+
+            if (nodeResult.snapshots().isEmpty()) {
+                printer.accept("\t\t" + NO_SNAPSHOTS);
+
+                continue;
+            }
+
+            // Sorted snapshots names for better user reading.
+            new TreeMap<>(nodeResult.snapshots()).forEach((snpName, snpInfo) -> {
+                printer.accept("\t\tSnapshot '%s': totalSize=%s (%db), created='%s' (epoch=%d)".formatted(

Review Comment:
   With the table format, the date format isn’t relevant. I believe we should discuss and choose a single common format, but not in this ticket. Here I’m sticking with the existing one.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4323559573


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4323532359


##########
modules/control-utility/src/test/java/org/apache/ignite/util/GridCommandHandlerListSnapshotTest.java:
##########
@@ -0,0 +1,348 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.util;
+
+import java.io.File;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Collection;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.DataStorageConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommand;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.G;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.lang.IgnitePredicate;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Test;
+import org.junit.runners.Parameterized.Parameter;
+import org.junit.runners.Parameterized.Parameters;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.cluster.ClusterState.ACTIVE;
+import static org.apache.ignite.internal.commandline.CommandHandler.EXIT_CODE_OK;
+import static org.apache.ignite.internal.processors.cache.persistence.snapshot.AbstractSnapshotSelfTest.snp;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Test for the command '--snapshot list'. */
+public class GridCommandHandlerListSnapshotTest extends GridCommandHandlerAbstractTest {
+    /** External storage path. */
+    private static final String EXT_STORAGE_PATH = "extStorage";
+
+    /** Flag to use {@link DataStorageConfiguration#setExtraSnapshotPaths(String...)}. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@code extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Consistent id postfix. */
+    private @Nullable String cstIdPostfix = "";
+
+    /** Flag setting the usage of a custom snapshot path. */
+    @Parameter(1)
+    public boolean customPath;
+
+    /** Flag setting the usage of dedicated, own node working directories. */
+    @Parameter(2)
+    public boolean separatedWorkDir;
+
+    /** Flag to add external server node after the snapshot creation. */
+    @Parameter(3)
+    public boolean addExtraSrvr;
+
+    /** Number of incremental snapshots to add to the main test snapshots. */
+    @Parameter(4)
+    public int incCnt;
+
+    /** */
+    @Parameters(name = "cmdHnd={0}, customPath={1}, ownWorkDir={2}, addExtraSrvr={3}, incCnt={4}")
+    public static Collection<?> parameters() {
+        return GridTestUtils.cartesianProduct(
+            commandHandlers(),
+            F.asList(false, true), // Custom snapshot path.
+            F.asList(false, true), // Separated (own) work directories.
+            F.asList(false, true), // Add a server node after the snapshot creation.
+            F.asList(0, 2) // Number of incremental snapshots.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        super.afterTest();
+
+        stopAllGrids();
+
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTest() throws Exception {
+        super.beforeTest();
+
+        /** Handy if test running is interrupted and {@link #afterTest()} isn't invoked. */
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
+
+        String workDir = separatedWorkDir
+            ? new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath()
+            : U.defaultWorkDirectory();
+
+        cfg.setWorkDirectory(workDir);
+
+        cfg.getDataStorageConfiguration().setWalCompactionEnabled(incCnt > 0);
+
+        if (extStorages) {
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                workDir + File.separator,
+                workDir + File.separator + EXT_STORAGE_PATH
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", EXT_STORAGE_PATH);
+        }
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override public String getTestIgniteInstanceName() {
+        return super.getTestIgniteInstanceName() + cstIdPostfix;
+    }
+
+    /** */
+    @Test
+    public void testNoSnapshots() throws Exception {
+        doTestSnapshotsLists(0, false, false);
+    }
+
+    /** */
+    @Test
+    public void testSingleSnapshot() throws Exception {
+        doTestSnapshotsLists(1, false, false);
+    }
+
+    /** */
+    @Test
+    public void testSeveralSnapshots() throws Exception {
+        doTestSnapshotsLists(3, false, false);
+    }
+
+    /** */
+    @Test
+    public void testOneNodeMisses() throws Exception {
+        // Doesn't matter here.
+//        assumeFalse(incCnt > 0);

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4375858233


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.Comparator;
+import java.util.Date;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Consumer;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.management.SystemViewCommand;
+import org.apache.ignite.internal.management.SystemViewTask;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+import static org.apache.ignite.internal.management.SystemViewTask.SimpleType.DATE;
+import static org.apache.ignite.internal.management.SystemViewTask.SimpleType.NUMBER;
+import static org.apache.ignite.internal.management.SystemViewTask.SimpleType.STRING;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots were found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes, including their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        // Flag whether the node's result has at least one incremental snapshot.
+        AtomicBoolean hasIncrementals = new AtomicBoolean();
+        // Flag whether the node's result has at least one external storage.
+        AtomicBoolean hasExtStorages = new AtomicBoolean();

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4363354739


##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListTest.java:
##########
@@ -0,0 +1,936 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.io.Serializable;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Stream;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.lang.IgniteFuture;
+import org.apache.ignite.plugin.AbstractTestPluginProvider;
+import org.apache.ignite.plugin.PluginConfiguration;
+import org.apache.ignite.plugin.PluginContext;
+import org.apache.ignite.plugin.PluginProvider;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runners.Parameterized;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.configuration.IgniteConfiguration.DFLT_SNAPSHOT_THREAD_POOL_SIZE;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.cartesianProduct;
+import static org.apache.ignite.testframework.GridTestUtils.runAsync;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Cluster-wide snapshot list procedure tests. */
+public class IgniteClusterSnapshotListTest extends AbstractSnapshotSelfTest {
+    /** Number of cache keys to pre-create at node startup. */
+    private static final int CACHE_KEYS_RANGE = 15;
+
+    /** Number of partitions of the test cache. */
+    private static final int CACHE_PARTITIONS_COUNT = 4;
+
+    /** Flag showing that the filesystem supports POSIX permissions. */
+    private static boolean posixPermissions;
+
+    /** Size of the snapshot utility thread pool. */
+    @Parameterized.Parameter(2)
+    public int snpThrdPoolSz;
+
+    /** */
+    private PluginProvider<PluginConfiguration> pluginProvider;
+
+    /** Flag to spread the test cache data over the external storages. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@link #extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Parameters. */
+    @Parameterized.Parameters(name = "encryption={0}, onlyPrimary={1}, snpThrdPoolSz={2}")
+    public static Collection<Object[]> params() {
+        return cartesianProduct(
+            encryptionParameters(), // Encryption.
+            F.asList(false, true), // Only primary.
+            F.asList(DFLT_SNAPSHOT_THREAD_POOL_SIZE, 1) // Snapshots thread pool size.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName)
+            .setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        if (pluginProvider != null)
+            cfg.setPluginProviders(pluginProvider);
+
+        if (extStorages) {
+            // External storage paths must be identical on all the nodes (a cache has the single storage paths
+            // setting). Thus, a shared work directory is used, like in GridCommandHandlerListSnapshotTest.
+            cfg.setWorkDirectory(U.defaultWorkDirectory());
+
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                U.defaultWorkDirectory() + File.separator,
+                U.defaultWorkDirectory() + File.separator + "extStorage"
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", "extStorage");
+        }
+
+        cfg.setSnapshotThreadPoolSize(snpThrdPoolSz);
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        super.cleanPersistenceDir();
+
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTestsStarted() throws Exception {
+        super.beforeTestsStarted();
+
+        File workDir = new File(U.defaultWorkDirectory());
+
+        assertTrue(workDir.exists());
+
+        try {
+            Files.getPosixFilePermissions(workDir.toPath());
+
+            posixPermissions = true;
+        }
+        catch (UnsupportedOperationException ignored) {
+            // No-op.
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected <K, V> CacheConfiguration<K, V> txCacheConfig(CacheConfiguration<K, V> ccfg) {
+        // Speeds up the tests.
+        ccfg = super.txCacheConfig(ccfg)
+            .setAffinity(new RendezvousAffinityFunction(false, CACHE_PARTITIONS_COUNT))
+            .setBackups(1);
+
+        if (extStorages) {
+            assert !F.isEmpty(extStoragePaths);
+
+            ccfg.setStoragePaths(extStoragePaths);
+        }
+
+        return ccfg;
+    }
+
+    /** */
+    @Test
+    public void testConcurrentCreation() throws Exception {
+        // The test uses the thread blocking and conditional waiting. Won't proceed with 1 thread.
+        assumeTrue(snpThrdPoolSz > 1);
+
+        int grids = 3;
+        int testNodeIdx = 1;
+        int testNodeOrder = testNodeIdx + 1;
+
+        CountDownLatch beginSnpCreation = new CountDownLatch(grids);
+        CountDownLatch proceedSnpCreation = new CountDownLatch(1);
+
+        // Delays snapshot creation after its metadata is written.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public <M extends Serializable> void storeSnapshotMeta(M meta, File smf) {
+                            super.storeSnapshotMeta(meta, smf);
+
+                            beginSnpCreation.countDown();
+
+                            if (((IgniteEx)ctx.grid()).localNode().order() == testNodeOrder) {
+                                try {
+                                    assertTrue(proceedSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        IgniteFuture<Void> createSnpFut = snp(grid(0)).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary);
+
+        assertTrue(beginSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        SnapshotListTaskResult lstOpRes = listSnapshots(grid(0));
+
+        int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+        assertEquals(grids, snpsCnt);
+
+        SnapshotFileTree testSft = new SnapshotFileTree(grid(testNodeIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(SnapshotListTask.calculateDirectorySize(testSft.root()) > 0L);
+
+        proceedSnpCreation.countDown();
+
+        createSnpFut.get(getTestTimeout());
+    }
+
+    /** */
+    @Test
+    public void testCompletelyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(true);
+    }
+
+    /** */
+    @Test
+    public void testPartlyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(false);
+    }
+
+    /** */
+    private void doTestDeletionAfterMetaRead(boolean completeDeletion) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        CountDownLatch metaReadBeginLatch = new CountDownLatch(grids);
+        CountDownLatch metaReadProceedLatch = new CountDownLatch(1);
+
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft, boolean failIfCantRead) {
+                            metaReadBeginLatch.countDown();
+
+                            if (cctx.localNode().order() == testGridIdx + 1) {
+                                try {
+                                    assertTrue(metaReadProceedLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        long testSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+        assertTrue(testSnpSz > 0);
+
+        IgniteInternalFuture<SnapshotListTaskResult> lstOpFut = runAsync(() -> listSnapshots(grid(0)));
+
+        assertTrue(metaReadBeginLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        if (completeDeletion)
+            assertTrue(testSnpFt.root().exists() && U.delete(testSnpFt.root()) && !testSnpFt.root().exists());
+        else
+            assertTrue(testSnpFt.nodeStorage().exists() && U.delete(testSnpFt.nodeStorage()) && !testSnpFt.nodeStorage().exists());
+
+        metaReadProceedLatch.countDown();
+
+        SnapshotListTaskResult lstOpRes = lstOpFut.get();
+
+        if (completeDeletion) {
+            int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+            assertEquals(grids - 1, snpsCnt);
+        }
+        else {
+            int snpsCnt = 0;
+            boolean victimNodeFound = false;
+
+            for (int i = 0; i < lstOpRes.nodesIds().length; i++) {
+                UUID nid = lstOpRes.nodesIds()[i];
+
+                Map<String, SnapshotListJobResult.SnapshotInfo> nodeSnps = lstOpRes.nodesSnapshots()[i].snapshots();
+
+                snpsCnt += nodeSnps.size();
+
+                if (nid.equals(grid(testGridIdx).localNode().id())) {
+                    victimNodeFound = true;
+
+                    assertEquals(1, nodeSnps.size());
+
+                    long curSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+                    assertTrue(curSnpSz < testSnpSz);
+                }
+            }
+
+            assertTrue(victimNodeFound);
+            assertEquals(grids, snpsCnt);
+        }
+    }
+
+    /** */
+    @Test
+    public void testMissingSnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(true);
+    }
+
+    /** */
+    @Test
+    public void testEmptySnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(false);
+    }
+
+    /** */
+    private void doTestWrongSnapshotPath(boolean missing) throws Exception {
+        // Doesn't matter here, speeds up the tests.
+        assumeFalse(snpThrdPoolSz < 2);

Review Comment:
   Done. Removed everywhere except the spot that intentionally blocks a thread for testing.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6374206366

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotCheckCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotDeleteCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotRestoreCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4323693321


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");
+
+            printer.accept("\tNode '%s' [uuid=%s]:".formatted(res.consistentIds()[nodeIdx], res.nodesIds()[nodeIdx]));
+
+            SnapshotListJobResult nodeResult = res.nodesSnapshots()[nodeIdx];
+
+            if (nodeResult.snapshots().isEmpty()) {
+                printer.accept("\t\t" + NO_SNAPSHOTS);
+
+                continue;
+            }
+
+            // Sorted snapshots names for better user reading.
+            new TreeMap<>(nodeResult.snapshots()).forEach((snpName, snpInfo) -> {
+                printer.accept("\t\tSnapshot '%s': totalSize=%s (%db), created='%s' (epoch=%d)".formatted(
+                    snpName,
+                    U.humanReadableByteCount(snpInfo.size()),
+                    snpInfo.size(),
+                    DATE_FORMATTER.format(Instant.ofEpochMilli(snpInfo.date())),
+                    snpInfo.date()
+                ));
+
+                SnapshotListJobResult.SnapshotInfo extStors = snpInfo.externalStorages();
+                SnapshotListJobResult.SnapshotInfo incs = snpInfo.incrementals();
+
+                if (extStors != null) {
+                    printer.accept("\t\t\texternal storages: cnt=%d, size=%s (%db)".formatted(
+                        extStors.number(),
+                        U.humanReadableByteCount(extStors.size()),
+                        extStors.size()
+                    ));
+                }
+
+                if (incs != null) {
+                    printer.accept("\t\t\tincremental snapshots: cnt=%d, size=%s (%db), modified='%s' (epoch=%d)".formatted(
+                        incs.number(),
+                        U.humanReadableByteCount(incs.size()),
+                        incs.size(),
+                        DATE_FORMATTER.format(Instant.ofEpochMilli(incs.date())),
+                        incs.date()
+                    ));
+                }
+            });
+        }
+
+        // Drop a line.

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324576118


##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListTest.java:
##########
@@ -0,0 +1,936 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.io.Serializable;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Stream;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.lang.IgniteFuture;
+import org.apache.ignite.plugin.AbstractTestPluginProvider;
+import org.apache.ignite.plugin.PluginConfiguration;
+import org.apache.ignite.plugin.PluginContext;
+import org.apache.ignite.plugin.PluginProvider;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runners.Parameterized;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.configuration.IgniteConfiguration.DFLT_SNAPSHOT_THREAD_POOL_SIZE;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.cartesianProduct;
+import static org.apache.ignite.testframework.GridTestUtils.runAsync;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Cluster-wide snapshot list procedure tests. */
+public class IgniteClusterSnapshotListTest extends AbstractSnapshotSelfTest {
+    /** Number of cache keys to pre-create at node startup. */
+    private static final int CACHE_KEYS_RANGE = 15;
+
+    /** Number of partitions of the test cache. */
+    private static final int CACHE_PARTITIONS_COUNT = 4;
+
+    /** Flag showing that the filesystem supports POSIX permissions. */
+    private static boolean posixPermissions;
+
+    /** Size of the snapshot utility thread pool. */
+    @Parameterized.Parameter(2)
+    public int snpThrdPoolSz;
+
+    /** */
+    private PluginProvider<PluginConfiguration> pluginProvider;
+
+    /** Flag to spread the test cache data over the external storages. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@link #extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Parameters. */
+    @Parameterized.Parameters(name = "encryption={0}, onlyPrimary={1}, snpThrdPoolSz={2}")
+    public static Collection<Object[]> params() {
+        return cartesianProduct(
+            encryptionParameters(), // Encryption.
+            F.asList(false, true), // Only primary.
+            F.asList(DFLT_SNAPSHOT_THREAD_POOL_SIZE, 1) // Snapshots thread pool size.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName)
+            .setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        if (pluginProvider != null)
+            cfg.setPluginProviders(pluginProvider);
+
+        if (extStorages) {
+            // External storage paths must be identical on all the nodes (a cache has the single storage paths
+            // setting). Thus, a shared work directory is used, like in GridCommandHandlerListSnapshotTest.
+            cfg.setWorkDirectory(U.defaultWorkDirectory());
+
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                U.defaultWorkDirectory() + File.separator,
+                U.defaultWorkDirectory() + File.separator + "extStorage"
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", "extStorage");
+        }
+
+        cfg.setSnapshotThreadPoolSize(snpThrdPoolSz);
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        super.cleanPersistenceDir();
+
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTestsStarted() throws Exception {
+        super.beforeTestsStarted();
+
+        File workDir = new File(U.defaultWorkDirectory());
+
+        assertTrue(workDir.exists());
+
+        try {
+            Files.getPosixFilePermissions(workDir.toPath());
+
+            posixPermissions = true;
+        }
+        catch (UnsupportedOperationException ignored) {
+            // No-op.
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected <K, V> CacheConfiguration<K, V> txCacheConfig(CacheConfiguration<K, V> ccfg) {
+        // Speeds up the tests.
+        ccfg = super.txCacheConfig(ccfg)
+            .setAffinity(new RendezvousAffinityFunction(false, CACHE_PARTITIONS_COUNT))
+            .setBackups(1);
+
+        if (extStorages) {
+            assert !F.isEmpty(extStoragePaths);
+
+            ccfg.setStoragePaths(extStoragePaths);
+        }
+
+        return ccfg;
+    }
+
+    /** */
+    @Test
+    public void testConcurrentCreation() throws Exception {
+        // The test uses the thread blocking and conditional waiting. Won't proceed with 1 thread.
+        assumeTrue(snpThrdPoolSz > 1);
+
+        int grids = 3;
+        int testNodeIdx = 1;
+        int testNodeOrder = testNodeIdx + 1;
+
+        CountDownLatch beginSnpCreation = new CountDownLatch(grids);
+        CountDownLatch proceedSnpCreation = new CountDownLatch(1);
+
+        // Delays snapshot creation after its metadata is written.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public <M extends Serializable> void storeSnapshotMeta(M meta, File smf) {
+                            super.storeSnapshotMeta(meta, smf);
+
+                            beginSnpCreation.countDown();
+
+                            if (((IgniteEx)ctx.grid()).localNode().order() == testNodeOrder) {
+                                try {
+                                    assertTrue(proceedSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        IgniteFuture<Void> createSnpFut = snp(grid(0)).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary);
+
+        assertTrue(beginSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        SnapshotListTaskResult lstOpRes = listSnapshots(grid(0));
+
+        int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+        assertEquals(grids, snpsCnt);
+
+        SnapshotFileTree testSft = new SnapshotFileTree(grid(testNodeIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(SnapshotListTask.calculateDirectorySize(testSft.root()) > 0L);
+
+        proceedSnpCreation.countDown();
+
+        createSnpFut.get(getTestTimeout());
+    }
+
+    /** */
+    @Test
+    public void testCompletelyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(true);
+    }
+
+    /** */
+    @Test
+    public void testPartlyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(false);
+    }
+
+    /** */
+    private void doTestDeletionAfterMetaRead(boolean completeDeletion) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        CountDownLatch metaReadBeginLatch = new CountDownLatch(grids);
+        CountDownLatch metaReadProceedLatch = new CountDownLatch(1);
+
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft, boolean failIfCantRead) {
+                            metaReadBeginLatch.countDown();
+
+                            if (cctx.localNode().order() == testGridIdx + 1) {
+                                try {
+                                    assertTrue(metaReadProceedLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        long testSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+        assertTrue(testSnpSz > 0);
+
+        IgniteInternalFuture<SnapshotListTaskResult> lstOpFut = runAsync(() -> listSnapshots(grid(0)));
+
+        assertTrue(metaReadBeginLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        if (completeDeletion)
+            assertTrue(testSnpFt.root().exists() && U.delete(testSnpFt.root()) && !testSnpFt.root().exists());
+        else
+            assertTrue(testSnpFt.nodeStorage().exists() && U.delete(testSnpFt.nodeStorage()) && !testSnpFt.nodeStorage().exists());
+
+        metaReadProceedLatch.countDown();
+
+        SnapshotListTaskResult lstOpRes = lstOpFut.get();
+
+        if (completeDeletion) {
+            int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+            assertEquals(grids - 1, snpsCnt);
+        }
+        else {
+            int snpsCnt = 0;
+            boolean victimNodeFound = false;
+
+            for (int i = 0; i < lstOpRes.nodesIds().length; i++) {
+                UUID nid = lstOpRes.nodesIds()[i];
+
+                Map<String, SnapshotListJobResult.SnapshotInfo> nodeSnps = lstOpRes.nodesSnapshots()[i].snapshots();
+
+                snpsCnt += nodeSnps.size();
+
+                if (nid.equals(grid(testGridIdx).localNode().id())) {
+                    victimNodeFound = true;
+
+                    assertEquals(1, nodeSnps.size());
+
+                    long curSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+                    assertTrue(curSnpSz < testSnpSz);
+                }
+            }
+
+            assertTrue(victimNodeFound);
+            assertEquals(grids, snpsCnt);
+        }
+    }
+
+    /** */
+    @Test
+    public void testMissingSnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(true);
+    }
+
+    /** */
+    @Test
+    public void testEmptySnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(false);
+    }
+
+    /** */
+    private void doTestWrongSnapshotPath(boolean missing) throws Exception {
+        // Doesn't matter here, speeds up the tests.
+        assumeFalse(snpThrdPoolSz < 2);

Review Comment:
   Why `snpThrdPoolSz = DFLT_SNAPSHOT_THREAD_POOL_SIZE;`? It would be configured.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6341930630

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsSingleMessage.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "zstan (via GitHub)" <gi...@apache.org>.
zstan merged PR #13666:
URL: https://github.com/apache/ignite/pull/13666


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4375845929


##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListTest.java:
##########
@@ -0,0 +1,918 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.io.Serializable;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Stream;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.io.GridFileUtils;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.lang.IgniteFuture;
+import org.apache.ignite.plugin.AbstractTestPluginProvider;
+import org.apache.ignite.plugin.PluginConfiguration;
+import org.apache.ignite.plugin.PluginContext;
+import org.apache.ignite.plugin.PluginProvider;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runners.Parameterized;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.configuration.IgniteConfiguration.DFLT_SNAPSHOT_THREAD_POOL_SIZE;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.cartesianProduct;
+import static org.apache.ignite.testframework.GridTestUtils.runAsync;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Cluster-wide snapshot list procedure tests. */
+public class IgniteClusterSnapshotListTest extends AbstractSnapshotSelfTest {
+    /** Number of cache keys to pre-create at node startup. */
+    private static final int CACHE_KEYS_RANGE = 15;
+
+    /** Number of partitions of the test cache. */
+    private static final int CACHE_PARTITIONS_COUNT = 4;
+
+    /** Flag showing that the filesystem supports POSIX permissions. */
+    private static boolean posixPermissions;
+
+    /** Size of the snapshot utility thread pool. */
+    @Parameterized.Parameter(2)
+    public int snpThrdPoolSz;
+
+    /** */
+    private PluginProvider<PluginConfiguration> pluginProvider;
+
+    /** Flag to spread the test cache data over external storages. */
+    private boolean extStorages;
+
+    /** Resolved external storage paths. {@code null} if {@link #extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Parameters. */
+    @Parameterized.Parameters(name = "encryption={0}, onlyPrimary={1}, snpThrdPoolSz={2}")
+    public static Collection<Object[]> params() {
+        return cartesianProduct(
+            encryptionParameters(), // Encryption.
+            F.asList(false, true), // Only primary.
+            F.asList(DFLT_SNAPSHOT_THREAD_POOL_SIZE, 1) // Snapshots thread pool size.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName)
+            .setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        if (pluginProvider != null)
+            cfg.setPluginProviders(pluginProvider);
+
+        if (extStorages) {
+            // External storage paths must be identical on all nodes (a cache has a single storage path setting).
+            // Thus, a shared work directory is used, like in GridCommandHandlerListSnapshotTest.
+            cfg.setWorkDirectory(U.defaultWorkDirectory());
+
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                U.defaultWorkDirectory() + File.separator,
+                U.defaultWorkDirectory() + File.separator + "extStorage"
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", "extStorage");
+        }
+
+        cfg.setSnapshotThreadPoolSize(snpThrdPoolSz);
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        super.cleanPersistenceDir();
+
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTestsStarted() throws Exception {
+        super.beforeTestsStarted();
+
+        File workDir = new File(U.defaultWorkDirectory());
+
+        assertTrue(workDir.exists());
+
+        try {
+            Files.getPosixFilePermissions(workDir.toPath());
+
+            posixPermissions = true;
+        }
+        catch (UnsupportedOperationException ignored) {
+            // No-op.
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected <K, V> CacheConfiguration<K, V> txCacheConfig(CacheConfiguration<K, V> ccfg) {
+        // Speeds up the tests.
+        ccfg = super.txCacheConfig(ccfg)
+            .setAffinity(new RendezvousAffinityFunction(false, CACHE_PARTITIONS_COUNT))
+            .setBackups(1);
+
+        if (extStorages) {
+            assert !F.isEmpty(extStoragePaths);
+
+            ccfg.setStoragePaths(extStoragePaths);
+        }
+
+        return ccfg;
+    }
+
+    /** */
+    @Test
+    public void testConcurrentCreation() throws Exception {
+        // The test relies on thread blocking and conditional waiting, so it does not run with a single thread.
+        assumeTrue(snpThrdPoolSz > 1);
+
+        int grids = 3;
+        int testNodeIdx = 1;
+        int testNodeOrder = testNodeIdx + 1;
+
+        CountDownLatch beginSnpCreation = new CountDownLatch(grids);
+        CountDownLatch proceedSnpCreation = new CountDownLatch(1);
+
+        // Delays snapshot creation after its metadata is written.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public <M extends Serializable> void storeSnapshotMeta(M meta, File smf) {
+                            super.storeSnapshotMeta(meta, smf);
+
+                            beginSnpCreation.countDown();
+
+                            if (((IgniteEx)ctx.grid()).localNode().order() == testNodeOrder) {
+                                try {
+                                    assertTrue(proceedSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        IgniteFuture<Void> createSnpFut = snp(grid(0)).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary);
+
+        assertTrue(beginSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        SnapshotListTaskResult lstOpRes = listSnapshots(grid(0));
+
+        int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+        assertEquals(grids, snpsCnt);
+
+        SnapshotFileTree testSft = new SnapshotFileTree(grid(testNodeIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(GridFileUtils.sizeOfDirectory(testSft.root()) > 0L);
+
+        proceedSnpCreation.countDown();
+
+        createSnpFut.get(getTestTimeout());
+    }
+
+    /** */
+    @Test
+    public void testCompletelyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(true);
+    }
+
+    /** */
+    @Test
+    public void testPartlyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(false);
+    }
+
+    /** */
+    private void doTestDeletionAfterMetaRead(boolean completeDeletion) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        CountDownLatch metaReadBeginLatch = new CountDownLatch(grids);
+        CountDownLatch metaReadProceedLatch = new CountDownLatch(1);
+
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft, boolean failIfCantRead) {
+                            metaReadBeginLatch.countDown();
+
+                            if (cctx.localNode().order() == testGridIdx + 1) {
+                                try {
+                                    assertTrue(metaReadProceedLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        long testSnpSz = GridFileUtils.sizeOfDirectory(testSnpFt.root());
+
+        assertTrue(testSnpSz > 0);
+
+        IgniteInternalFuture<SnapshotListTaskResult> lstOpFut = runAsync(() -> listSnapshots(grid(0)));
+
+        assertTrue(metaReadBeginLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        if (completeDeletion)
+            assertTrue(testSnpFt.root().exists() && U.delete(testSnpFt.root()) && !testSnpFt.root().exists());
+        else
+            assertTrue(testSnpFt.nodeStorage().exists() && U.delete(testSnpFt.nodeStorage()) && !testSnpFt.nodeStorage().exists());
+
+        metaReadProceedLatch.countDown();
+
+        SnapshotListTaskResult lstOpRes = lstOpFut.get();
+
+        if (completeDeletion) {
+            int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+            assertEquals(grids - 1, snpsCnt);
+        }
+        else {
+            int snpsCnt = 0;
+            boolean victimNodeFound = false;
+
+            for (int i = 0; i < lstOpRes.nodesIds().length; i++) {
+                UUID nid = lstOpRes.nodesIds()[i];
+
+                Map<String, SnapshotListJobResult.SnapshotInfo> nodeSnps = lstOpRes.nodesSnapshots()[i].snapshots();
+
+                snpsCnt += nodeSnps.size();
+
+                if (nid.equals(grid(testGridIdx).localNode().id())) {
+                    victimNodeFound = true;
+
+                    assertEquals(1, nodeSnps.size());
+
+                    long curSnpSz = GridFileUtils.sizeOfDirectory(testSnpFt.root());
+
+                    assertTrue(curSnpSz < testSnpSz);
+                }
+            }
+
+            assertTrue(victimNodeFound);
+            assertEquals(grids, snpsCnt);
+        }
+    }
+
+    /** */
+    @Test
+    public void testEmptySnapshotPath() throws Exception {
+        startGrids(3);
+
+        File path = new File(U.defaultWorkDirectory(), "not_snapshots");
+
+        SnapshotListJobResult[] listOperationRes = listSnapshots(grid(0), path.getAbsolutePath()).nodesSnapshots();
+
+        int cnt = Stream.of(listOperationRes).mapToInt(nodeRes -> nodeRes.snapshots().size()).sum();
+
+        assertEquals(0, cnt);
+    }
+
+    /** */
+    @Test
+    public void testDefaultSnapshotPath() throws Exception {
+        doTestSnapshotPath(null);
+    }
+
+    /** */
+    @Test
+    @Ignore("https://issues.apache.org/jira/browse/IGNITE-29126")
+    public void testRelativeSnapshotPath() throws Exception {
+        doTestSnapshotPath("ex_snapshots");
+    }
+
+    /** */
+    @Test
+    public void testAbsoluteSnapshotPath() throws Exception {
+        doTestSnapshotPath(new File(U.defaultWorkDirectory(), "ex_snapshots").getAbsolutePath());
+    }
+
+    /** */
+    private void doTestSnapshotPath(@Nullable String path) throws Exception {
+        int grids = 3;
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        snp(grid(0)).createSnapshot(SNAPSHOT_NAME, path, false, onlyPrimary).get(getTestTimeout());
+
+        SnapshotListJobResult[] lstOpRes = listSnapshots(grid(0), path).nodesSnapshots();
+
+        int cnt = Stream.of(lstOpRes).mapToInt(nodeRes -> nodeRes.snapshots().size()).sum();
+
+        assertEquals(grids, cnt);
+    }
+
+    /** */
+    @Test
+    public void testMissingMeta() throws Exception {
+        doTestWithWrongMeta(false);
+    }
+
+    /** */
+    @Test
+    public void testCorruptedMeta() throws Exception {
+        doTestWithWrongMeta(true);
+    }
+
+    /**
+     * Tests snapshot lists when one snapshot metadata cannot be read.
+     *
+     * @param corruptFile If {@code true}, corrupts metadata. Otherwise, deletes metadata.
+     */
+    private void doTestWithWrongMeta(boolean corruptFile) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(testSnpFt.meta().exists());
+
+        if (corruptFile) {
+            try (RandomAccessFile raf = new RandomAccessFile(testSnpFt.meta(), "rw")) {
+                raf.write(UUID.randomUUID().toString().getBytes());
+            }
+        }
+        else
+            assertTrue(testSnpFt.meta().delete() && !testSnpFt.meta().exists());
+
+        SnapshotListTaskResult res = listSnapshots(grid(0));
+
+        int foundSnpsCnt = 0;
+        boolean victimNodeFound = false;
+
+        for (int i = 0; i < res.nodesIds().length; i++) {
+            UUID nid = res.nodesIds()[i];
+
+            foundSnpsCnt += res.nodesSnapshots()[i].snapshots().size();
+
+            if (nid.equals(grid(testGridIdx).localNode().id())) {
+                victimNodeFound = true;
+
+                assertTrue(res.nodesSnapshots()[i].snapshots().isEmpty());
+            }
+        }
+
+        assertTrue(victimNodeFound);
+        assertEquals(grids - 1, foundSnpsCnt);
+    }
+
+    /** */
+    @Test
+    public void testMissingIncrementalMeta() throws Exception {
+        doTestWithWrongIncrementalMeta(false);
+    }
+
+    /** */
+    @Test
+    public void testCorruptedIncrementalMeta() throws Exception {
+        doTestWithWrongIncrementalMeta(true);
+    }
+
+    /**
+     * Tests snapshot list when incremental snapshot metadata cannot be read.
+     * The main snapshot should still be listed, but without incremental info on the affected node.
+     *
+     * @param corruptFile If {@code true}, corrupts metadata. Otherwise, deletes metadata.
+     */
+    private void doTestWithWrongIncrementalMeta(boolean corruptFile) throws Exception {
+        // Incremental snapshots do not support the only-primary mode or encryption.
+        assumeFalse(onlyPrimary || encryption);
+
+        int grids = 3;
+        int testGridIdx = 1;
+        int incsCnt = 3;
+
+        IgniteEx ig = startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        snp(ig).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary).get(getTestTimeout());
+
+        for (int i = 0; i < incsCnt; ++i) {
+            try (IgniteDataStreamer<Integer, Integer> ds = grid(0).dataStreamer(DEFAULT_CACHE_NAME)) {
+                for (int kv = (i + 1) * CACHE_KEYS_RANGE; kv < (i + 1) * CACHE_KEYS_RANGE * 2; ++kv)
+                    ds.addData(kv, kv);
+            }
+
+            snp(ig).createSnapshot(SNAPSHOT_NAME, null, true, onlyPrimary).get(getTestTimeout());
+        }
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+        SnapshotFileTree.IncrementalSnapshotFileTree incFt = testSnpFt.incrementalSnapshotFileTree(1);
+        File incMeta = incFt.meta();
+
+        assertTrue(incMeta.exists());
+
+        if (corruptFile) {
+            try (RandomAccessFile raf = new RandomAccessFile(incMeta, "rw")) {
+                raf.write(UUID.randomUUID().toString().getBytes());
+            }
+        }
+        else
+            assertTrue(incMeta.delete() && !incMeta.exists());
+
+        SnapshotListTaskResult res = listSnapshots(grid(0));
+
+        for (int i = 0; i < res.nodesIds().length; i++) {
+            UUID nid = res.nodesIds()[i];
+
+            Map<String, SnapshotListJobResult.SnapshotInfo> snps = res.nodesSnapshots()[i].snapshots();
+
+            assertEquals(1, snps.size());
+
+            SnapshotListJobResult.SnapshotInfo info = snps.get(SNAPSHOT_NAME);
+
+            assertNotNull(info);
+            assertNotNull(info.incrementals());
+
+            if (nid.equals(grid(testGridIdx).localNode().id()))
+                assertEquals(incsCnt - 1, info.incrementals().number().intValue());
+            else
+                assertEquals(incsCnt, info.incrementals().number().intValue());
+        }
+    }
+
+    /**
+     * Test snapshot list operation when a node can't read some snapshot part due to insufficient permissions.
+     * That is, a test node can read the snapshot metadata but cannot read some of the snapshot data.
+     */
+    @Test
+    public void testDeniedPermissions() throws Exception {
+        assumeTrue(posixPermissions);
+        // We rely on the snapshot sizes here. Better to avoid empty data nodes not to become flaky.
+        assumeFalse(onlyPrimary);
+
+        int grids = 3;
+        int testGridIdx = 1;
+
+        // Permissions to restore.
+        Map<Path, Set<PosixFilePermission>> oldPerms = new ConcurrentHashMap<>();
+        // The 'change permissions' flag.
+        AtomicBoolean changePermissions = new AtomicBoolean(true);
+
+        // Deny reading the whole snapshot part of one node.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft,
+                            boolean failIfCantRead) {
+                            if (changePermissions.get() && ctx.localNode().order() == testGridIdx + 1) {
+                                File victimDir = sft.nodeStorage();
+
+                                assertTrue(victimDir.exists());
+                                assertTrue(victimDir.isDirectory());
+
+                                // Ensure the blocked snapshot part has a non-zero size; the sizes are compared later.
+                                try {
+                                    assertTrue(GridFileUtils.sizeOfDirectory(victimDir) > 0L);
+                                }
+                                catch (IOException e) {
+                                    throw new IllegalStateException(e);
+                                }
+
+                                Path victimDirPath = victimDir.toPath();
+
+                                try {
+                                    Set<PosixFilePermission> perms = Files.getPosixFilePermissions(victimDirPath);
+
+                                    assertFalse(perms.isEmpty());
+
+                                    // Deny reading.
+                                    Files.setPosixFilePermissions(victimDirPath, PosixFilePermissions.fromString("---------"));
+
+                                    // Save actual permissions to restore.
+                                    oldPerms.put(victimDirPath, perms);
+                                }
+                                catch (Exception e) {
+                                    throw new IgniteException("Unable to set the test POSIX permissions.", e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, true);
+
+        UUID testNodeId = grid(testGridIdx).localNode().id();
+
+        // Snapshot size with restricted permissions.
+        long testSize0 = 0L;
+        // Snapshot size with normal permissions.
+        long testSize1 = 0L;
+
+        SnapshotListTaskResult listTaskRes;
+
+        try {
+            // First run.
+            listTaskRes = listSnapshots(grid(0));
+
+            assertFalse(oldPerms.isEmpty());
+
+            for (int i = 0; i < listTaskRes.nodesIds().length; i++) {
+                UUID nodeId = listTaskRes.nodesIds()[i];
+
+                // Store the partly read snapshot size.
+                if (nodeId.equals(testNodeId)) {
+                    Map<String, SnapshotListJobResult.SnapshotInfo> snps = listTaskRes.nodesSnapshots()[i].snapshots();
+
+                    assertEquals(1, snps.size());
+
+                    testSize0 = snps.get(SNAPSHOT_NAME).size();
+                }
+            }
+
+            // Ensure that we've found and read the snapshot on the test node.
+            assertTrue(testSize0 > 0L);
+        }
+        finally {
+            // Restore the permissions in any case.
+            oldPerms.forEach((path, perms) -> {
+                try {
+                    Files.setPosixFilePermissions(path, perms);
+                }
+                catch (IOException e) {
+                    throw new IllegalStateException(e);
+                }
+            });
+        }
+
+        // Relaunch the operation.
+        changePermissions.set(false);
+
+        listTaskRes = listSnapshots(grid(0));
+
+        for (int i = 0; i < listTaskRes.nodesIds().length; i++) {
+            UUID nodeId = listTaskRes.nodesIds()[i];
+
+            // Store the fully read snapshot size.
+            if (nodeId.equals(testNodeId)) {
+                Map<String, SnapshotListJobResult.SnapshotInfo> snps = listTaskRes.nodesSnapshots()[i].snapshots();
+
+                assertEquals(1, snps.size());
+
+                testSize1 = snps.get(SNAPSHOT_NAME).size();
+            }
+        }
+
+        // Ensure that the newly calculated size is greater than in the previous run.
+        assertTrue(testSize1 > 0L);
+        assertTrue(testSize1 > testSize0);
+    }
+
+    /** */
+    @Test
+    public void testSnapshotListsDates() throws Exception {
+        // Incremental snapshots do not support the only-primary mode or encryption.
+        assumeFalse(onlyPrimary || encryption);
+
+        int grids = 3;
+
+        IgniteEx ig = startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        long time0 = U.currentTimeMillis();
+
+        // Wait so that the timestamps differ.
+        U.sleep(300L);

Review Comment:
   Done. Also reduced it the command test of the order. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4375845929


##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListTest.java:
##########
@@ -0,0 +1,918 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.io.Serializable;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Stream;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.io.GridFileUtils;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.lang.IgniteFuture;
+import org.apache.ignite.plugin.AbstractTestPluginProvider;
+import org.apache.ignite.plugin.PluginConfiguration;
+import org.apache.ignite.plugin.PluginContext;
+import org.apache.ignite.plugin.PluginProvider;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runners.Parameterized;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.configuration.IgniteConfiguration.DFLT_SNAPSHOT_THREAD_POOL_SIZE;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.cartesianProduct;
+import static org.apache.ignite.testframework.GridTestUtils.runAsync;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Cluster-wide snapshot list procedure tests. */
+public class IgniteClusterSnapshotListTest extends AbstractSnapshotSelfTest {
+    /** Number of cache keys to pre-create at node startup. */
+    private static final int CACHE_KEYS_RANGE = 15;
+
+    /** Number of partitions of the test cache. */
+    private static final int CACHE_PARTITIONS_COUNT = 4;
+
+    /** Flag showing that the filesystem supports POSIX permissions. */
+    private static boolean posixPermissions;
+
+    /** Size of the snapshot utility thread pool. */
+    @Parameterized.Parameter(2)
+    public int snpThrdPoolSz;
+
+    /** */
+    private PluginProvider<PluginConfiguration> pluginProvider;
+
+    /** Flag to spread the test cache data over external storages. */
+    private boolean extStorages;
+
+    /** Resolved external storage paths. {@code null} if {@link #extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Parameters. */
+    @Parameterized.Parameters(name = "encryption={0}, onlyPrimary={1}, snpThrdPoolSz={2}")
+    public static Collection<Object[]> params() {
+        return cartesianProduct(
+            encryptionParameters(), // Encryption.
+            F.asList(false, true), // Only primary.
+            F.asList(DFLT_SNAPSHOT_THREAD_POOL_SIZE, 1) // Snapshots thread pool size.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName)
+            .setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        if (pluginProvider != null)
+            cfg.setPluginProviders(pluginProvider);
+
+        if (extStorages) {
+            // External storage paths must be identical on all nodes (a cache has a single storage path setting).
+            // Thus, a shared work directory is used, like in GridCommandHandlerListSnapshotTest.
+            cfg.setWorkDirectory(U.defaultWorkDirectory());
+
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                U.defaultWorkDirectory() + File.separator,
+                U.defaultWorkDirectory() + File.separator + "extStorage"
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", "extStorage");
+        }
+
+        cfg.setSnapshotThreadPoolSize(snpThrdPoolSz);
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        super.cleanPersistenceDir();
+
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTestsStarted() throws Exception {
+        super.beforeTestsStarted();
+
+        File workDir = new File(U.defaultWorkDirectory());
+
+        assertTrue(workDir.exists());
+
+        try {
+            Files.getPosixFilePermissions(workDir.toPath());
+
+            posixPermissions = true;
+        }
+        catch (UnsupportedOperationException ignored) {
+            // No-op.
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected <K, V> CacheConfiguration<K, V> txCacheConfig(CacheConfiguration<K, V> ccfg) {
+        // Speeds up the tests.
+        ccfg = super.txCacheConfig(ccfg)
+            .setAffinity(new RendezvousAffinityFunction(false, CACHE_PARTITIONS_COUNT))
+            .setBackups(1);
+
+        if (extStorages) {
+            assert !F.isEmpty(extStoragePaths);
+
+            ccfg.setStoragePaths(extStoragePaths);
+        }
+
+        return ccfg;
+    }
+
+    /** */
+    @Test
+    public void testConcurrentCreation() throws Exception {
+        // The test relies on thread blocking and conditional waiting, so it does not run with a single thread.
+        assumeTrue(snpThrdPoolSz > 1);
+
+        int grids = 3;
+        int testNodeIdx = 1;
+        int testNodeOrder = testNodeIdx + 1;
+
+        CountDownLatch beginSnpCreation = new CountDownLatch(grids);
+        CountDownLatch proceedSnpCreation = new CountDownLatch(1);
+
+        // Delays snapshot creation after its metadata is written.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public <M extends Serializable> void storeSnapshotMeta(M meta, File smf) {
+                            super.storeSnapshotMeta(meta, smf);
+
+                            beginSnpCreation.countDown();
+
+                            if (((IgniteEx)ctx.grid()).localNode().order() == testNodeOrder) {
+                                try {
+                                    assertTrue(proceedSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        IgniteFuture<Void> createSnpFut = snp(grid(0)).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary);
+
+        assertTrue(beginSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        SnapshotListTaskResult lstOpRes = listSnapshots(grid(0));
+
+        int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+        assertEquals(grids, snpsCnt);
+
+        SnapshotFileTree testSft = new SnapshotFileTree(grid(testNodeIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(GridFileUtils.sizeOfDirectory(testSft.root()) > 0L);
+
+        proceedSnpCreation.countDown();
+
+        createSnpFut.get(getTestTimeout());
+    }
+
+    /** */
+    @Test
+    public void testCompletelyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(true);
+    }
+
+    /** */
+    @Test
+    public void testPartlyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(false);
+    }
+
+    /** */
+    private void doTestDeletionAfterMetaRead(boolean completeDeletion) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        CountDownLatch metaReadBeginLatch = new CountDownLatch(grids);
+        CountDownLatch metaReadProceedLatch = new CountDownLatch(1);
+
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft, boolean failIfCantRead) {
+                            metaReadBeginLatch.countDown();
+
+                            if (cctx.localNode().order() == testGridIdx + 1) {
+                                try {
+                                    assertTrue(metaReadProceedLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        long testSnpSz = GridFileUtils.sizeOfDirectory(testSnpFt.root());
+
+        assertTrue(testSnpSz > 0);
+
+        IgniteInternalFuture<SnapshotListTaskResult> lstOpFut = runAsync(() -> listSnapshots(grid(0)));
+
+        assertTrue(metaReadBeginLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        if (completeDeletion)
+            assertTrue(testSnpFt.root().exists() && U.delete(testSnpFt.root()) && !testSnpFt.root().exists());
+        else
+            assertTrue(testSnpFt.nodeStorage().exists() && U.delete(testSnpFt.nodeStorage()) && !testSnpFt.nodeStorage().exists());
+
+        metaReadProceedLatch.countDown();
+
+        SnapshotListTaskResult lstOpRes = lstOpFut.get();
+
+        if (completeDeletion) {
+            int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+            assertEquals(grids - 1, snpsCnt);
+        }
+        else {
+            int snpsCnt = 0;
+            boolean victimNodeFound = false;
+
+            for (int i = 0; i < lstOpRes.nodesIds().length; i++) {
+                UUID nid = lstOpRes.nodesIds()[i];
+
+                Map<String, SnapshotListJobResult.SnapshotInfo> nodeSnps = lstOpRes.nodesSnapshots()[i].snapshots();
+
+                snpsCnt += nodeSnps.size();
+
+                if (nid.equals(grid(testGridIdx).localNode().id())) {
+                    victimNodeFound = true;
+
+                    assertEquals(1, nodeSnps.size());
+
+                    long curSnpSz = GridFileUtils.sizeOfDirectory(testSnpFt.root());
+
+                    assertTrue(curSnpSz < testSnpSz);
+                }
+            }
+
+            assertTrue(victimNodeFound);
+            assertEquals(grids, snpsCnt);
+        }
+    }
+
+    /** */
+    @Test
+    public void testEmptySnapshotPath() throws Exception {
+        startGrids(3);
+
+        File path = new File(U.defaultWorkDirectory(), "not_snapshots");
+
+        SnapshotListJobResult[] listOperationRes = listSnapshots(grid(0), path.getAbsolutePath()).nodesSnapshots();
+
+        int cnt = Stream.of(listOperationRes).mapToInt(nodeRes -> nodeRes.snapshots().size()).sum();
+
+        assertEquals(0, cnt);
+    }
+
+    /** */
+    @Test
+    public void testDefaultSnapshotPath() throws Exception {
+        doTestSnapshotPath(null);
+    }
+
+    /** */
+    @Test
+    @Ignore("https://issues.apache.org/jira/browse/IGNITE-29126")
+    public void testRelativeSnapshotPath() throws Exception {
+        doTestSnapshotPath("ex_snapshots");
+    }
+
+    /** */
+    @Test
+    public void testAbsoluteSnapshotPath() throws Exception {
+        doTestSnapshotPath(new File(U.defaultWorkDirectory(), "ex_snapshots").getAbsolutePath());
+    }
+
+    /** */
+    private void doTestSnapshotPath(@Nullable String path) throws Exception {
+        int grids = 3;
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        snp(grid(0)).createSnapshot(SNAPSHOT_NAME, path, false, onlyPrimary).get(getTestTimeout());
+
+        SnapshotListJobResult[] lstOpRes = listSnapshots(grid(0), path).nodesSnapshots();
+
+        int cnt = Stream.of(lstOpRes).mapToInt(nodeRes -> nodeRes.snapshots().size()).sum();
+
+        assertEquals(grids, cnt);
+    }
+
+    /** */
+    @Test
+    public void testMissingMeta() throws Exception {
+        doTestWithWrongMeta(false);
+    }
+
+    /** */
+    @Test
+    public void testCorruptedMeta() throws Exception {
+        doTestWithWrongMeta(true);
+    }
+
+    /**
+     * Tests snapshot lists when one snapshot metadata cannot be read.
+     *
+     * @param corruptFile If {@code true}, corrupts metadata. Otherwise, deletes metadata.
+     */
+    private void doTestWithWrongMeta(boolean corruptFile) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(testSnpFt.meta().exists());
+
+        if (corruptFile) {
+            try (RandomAccessFile raf = new RandomAccessFile(testSnpFt.meta(), "rw")) {
+                raf.write(UUID.randomUUID().toString().getBytes());
+            }
+        }
+        else
+            assertTrue(testSnpFt.meta().delete() && !testSnpFt.meta().exists());
+
+        SnapshotListTaskResult res = listSnapshots(grid(0));
+
+        int foundSnpsCnt = 0;
+        boolean victimNodeFound = false;
+
+        for (int i = 0; i < res.nodesIds().length; i++) {
+            UUID nid = res.nodesIds()[i];
+
+            foundSnpsCnt += res.nodesSnapshots()[i].snapshots().size();
+
+            if (nid.equals(grid(testGridIdx).localNode().id())) {
+                victimNodeFound = true;
+
+                assertTrue(res.nodesSnapshots()[i].snapshots().isEmpty());
+            }
+        }
+
+        assertTrue(victimNodeFound);
+        assertEquals(grids - 1, foundSnpsCnt);
+    }
+
+    /** */
+    @Test
+    public void testMissingIncrementalMeta() throws Exception {
+        doTestWithWrongIncrementalMeta(false);
+    }
+
+    /** */
+    @Test
+    public void testCorruptedIncrementalMeta() throws Exception {
+        doTestWithWrongIncrementalMeta(true);
+    }
+
+    /**
+     * Tests snapshot list when incremental snapshot metadata cannot be read.
+     * The main snapshot should still be listed, but without incremental info on the affected node.
+     *
+     * @param corruptFile If {@code true}, corrupts metadata. Otherwise, deletes metadata.
+     */
+    private void doTestWithWrongIncrementalMeta(boolean corruptFile) throws Exception {
+        // Incremental snapshots do not support the only-primary mode or encryption.
+        assumeFalse(onlyPrimary || encryption);
+
+        int grids = 3;
+        int testGridIdx = 1;
+        int incsCnt = 3;
+
+        IgniteEx ig = startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        snp(ig).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary).get(getTestTimeout());
+
+        for (int i = 0; i < incsCnt; ++i) {
+            try (IgniteDataStreamer<Integer, Integer> ds = grid(0).dataStreamer(DEFAULT_CACHE_NAME)) {
+                for (int kv = (i + 1) * CACHE_KEYS_RANGE; kv < (i + 1) * CACHE_KEYS_RANGE * 2; ++kv)
+                    ds.addData(kv, kv);
+            }
+
+            snp(ig).createSnapshot(SNAPSHOT_NAME, null, true, onlyPrimary).get(getTestTimeout());
+        }
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+        SnapshotFileTree.IncrementalSnapshotFileTree incFt = testSnpFt.incrementalSnapshotFileTree(1);
+        File incMeta = incFt.meta();
+
+        assertTrue(incMeta.exists());
+
+        if (corruptFile) {
+            try (RandomAccessFile raf = new RandomAccessFile(incMeta, "rw")) {
+                raf.write(UUID.randomUUID().toString().getBytes());
+            }
+        }
+        else
+            assertTrue(incMeta.delete() && !incMeta.exists());
+
+        SnapshotListTaskResult res = listSnapshots(grid(0));
+
+        for (int i = 0; i < res.nodesIds().length; i++) {
+            UUID nid = res.nodesIds()[i];
+
+            Map<String, SnapshotListJobResult.SnapshotInfo> snps = res.nodesSnapshots()[i].snapshots();
+
+            assertEquals(1, snps.size());
+
+            SnapshotListJobResult.SnapshotInfo info = snps.get(SNAPSHOT_NAME);
+
+            assertNotNull(info);
+            assertNotNull(info.incrementals());
+
+            if (nid.equals(grid(testGridIdx).localNode().id()))
+                assertEquals(incsCnt - 1, info.incrementals().number().intValue());
+            else
+                assertEquals(incsCnt, info.incrementals().number().intValue());
+        }
+    }
+
+    /**
+     * Test snapshot list operation when a node can't read some snapshot part due to insufficient permissions.
+     * That is, a test node can read the snapshot metadata but cannot read some of the snapshot data.
+     */
+    @Test
+    public void testDeniedPermissions() throws Exception {
+        assumeTrue(posixPermissions);
+        // We rely on the snapshot sizes here. Better to avoid empty data nodes not to become flaky.
+        assumeFalse(onlyPrimary);
+
+        int grids = 3;
+        int testGridIdx = 1;
+
+        // Permissions to restore.
+        Map<Path, Set<PosixFilePermission>> oldPerms = new ConcurrentHashMap<>();
+        // The 'change permissions' flag.
+        AtomicBoolean changePermissions = new AtomicBoolean(true);
+
+        // Deny reading the whole snapshot part of one node.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft,
+                            boolean failIfCantRead) {
+                            if (changePermissions.get() && ctx.localNode().order() == testGridIdx + 1) {
+                                File victimDir = sft.nodeStorage();
+
+                                assertTrue(victimDir.exists());
+                                assertTrue(victimDir.isDirectory());
+
+                                // Ensure the blocked snapshot part has a non-zero size; the sizes are compared later.
+                                try {
+                                    assertTrue(GridFileUtils.sizeOfDirectory(victimDir) > 0L);
+                                }
+                                catch (IOException e) {
+                                    throw new IllegalStateException(e);
+                                }
+
+                                Path victimDirPath = victimDir.toPath();
+
+                                try {
+                                    Set<PosixFilePermission> perms = Files.getPosixFilePermissions(victimDirPath);
+
+                                    assertFalse(perms.isEmpty());
+
+                                    // Deny reading.
+                                    Files.setPosixFilePermissions(victimDirPath, PosixFilePermissions.fromString("---------"));
+
+                                    // Save actual permissions to restore.
+                                    oldPerms.put(victimDirPath, perms);
+                                }
+                                catch (Exception e) {
+                                    throw new IgniteException("Unable to set the test POSIX permissions.", e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, true);
+
+        UUID testNodeId = grid(testGridIdx).localNode().id();
+
+        // Snapshot size with restricted permissions.
+        long testSize0 = 0L;
+        // Snapshot size with normal permissions.
+        long testSize1 = 0L;
+
+        SnapshotListTaskResult listTaskRes;
+
+        try {
+            // First run.
+            listTaskRes = listSnapshots(grid(0));
+
+            assertFalse(oldPerms.isEmpty());
+
+            for (int i = 0; i < listTaskRes.nodesIds().length; i++) {
+                UUID nodeId = listTaskRes.nodesIds()[i];
+
+                // Store the partly read snapshot size.
+                if (nodeId.equals(testNodeId)) {
+                    Map<String, SnapshotListJobResult.SnapshotInfo> snps = listTaskRes.nodesSnapshots()[i].snapshots();
+
+                    assertEquals(1, snps.size());
+
+                    testSize0 = snps.get(SNAPSHOT_NAME).size();
+                }
+            }
+
+            // Ensure that we've found and read the snapshot on the test node.
+            assertTrue(testSize0 > 0L);
+        }
+        finally {
+            // Restore the permissions in any case.
+            oldPerms.forEach((path, perms) -> {
+                try {
+                    Files.setPosixFilePermissions(path, perms);
+                }
+                catch (IOException e) {
+                    throw new IllegalStateException(e);
+                }
+            });
+        }
+
+        // Relaunch the operation.
+        changePermissions.set(false);
+
+        listTaskRes = listSnapshots(grid(0));
+
+        for (int i = 0; i < listTaskRes.nodesIds().length; i++) {
+            UUID nodeId = listTaskRes.nodesIds()[i];
+
+            // Store the fully read snapshot size.
+            if (nodeId.equals(testNodeId)) {
+                Map<String, SnapshotListJobResult.SnapshotInfo> snps = listTaskRes.nodesSnapshots()[i].snapshots();
+
+                assertEquals(1, snps.size());
+
+                testSize1 = snps.get(SNAPSHOT_NAME).size();
+            }
+        }
+
+        // Ensure that the newly calculated size is greater than in the previous run.
+        assertTrue(testSize1 > 0L);
+        assertTrue(testSize1 > testSize0);
+    }
+
+    /** */
+    @Test
+    public void testSnapshotListsDates() throws Exception {
+        // Incremental snapshots do not support the only-primary mode or encryption.
+        assumeFalse(onlyPrimary || encryption);
+
+        int grids = 3;
+
+        IgniteEx ig = startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        long time0 = U.currentTimeMillis();
+
+        // Wait so that the timestamps differ.
+        U.sleep(300L);

Review Comment:
   Done. Reduced it the command test of the order. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4323732225


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',

Review Comment:
   Good question. To me, it is better to do as much work as we can and deliver the most complete result. Unfortunately, this is not a common practice here in Apache Ignite 2 (in the computation, tasks and jobs). Normally, we use a fail-fast approach. However, a failure isn't expected to happen often. Such exception handling is acceptable too. WDYT?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4323618737


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");
+
+            printer.accept("\tNode '%s' [uuid=%s]:".formatted(res.consistentIds()[nodeIdx], res.nodesIds()[nodeIdx]));
+
+            SnapshotListJobResult nodeResult = res.nodesSnapshots()[nodeIdx];
+
+            if (nodeResult.snapshots().isEmpty()) {
+                printer.accept("\t\t" + NO_SNAPSHOTS);
+
+                continue;
+            }
+
+            // Sorted snapshots names for better user reading.

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324589320


##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListTest.java:
##########
@@ -0,0 +1,936 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.io.Serializable;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Stream;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.lang.IgniteFuture;
+import org.apache.ignite.plugin.AbstractTestPluginProvider;
+import org.apache.ignite.plugin.PluginConfiguration;
+import org.apache.ignite.plugin.PluginContext;
+import org.apache.ignite.plugin.PluginProvider;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runners.Parameterized;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.configuration.IgniteConfiguration.DFLT_SNAPSHOT_THREAD_POOL_SIZE;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.cartesianProduct;
+import static org.apache.ignite.testframework.GridTestUtils.runAsync;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Cluster-wide snapshot list procedure tests. */
+public class IgniteClusterSnapshotListTest extends AbstractSnapshotSelfTest {
+    /** Number of cache keys to pre-create at node startup. */
+    private static final int CACHE_KEYS_RANGE = 15;
+
+    /** Number of partitions of the test cache. */
+    private static final int CACHE_PARTITIONS_COUNT = 4;
+
+    /** Flag showing that the filesystem supports POSIX permissions. */
+    private static boolean posixPermissions;
+
+    /** Size of the snapshot utility thread pool. */
+    @Parameterized.Parameter(2)
+    public int snpThrdPoolSz;
+
+    /** */
+    private PluginProvider<PluginConfiguration> pluginProvider;
+
+    /** Flag to spread the test cache data over the external storages. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@link #extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Parameters. */
+    @Parameterized.Parameters(name = "encryption={0}, onlyPrimary={1}, snpThrdPoolSz={2}")
+    public static Collection<Object[]> params() {
+        return cartesianProduct(
+            encryptionParameters(), // Encryption.
+            F.asList(false, true), // Only primary.
+            F.asList(DFLT_SNAPSHOT_THREAD_POOL_SIZE, 1) // Snapshots thread pool size.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName)
+            .setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        if (pluginProvider != null)
+            cfg.setPluginProviders(pluginProvider);
+
+        if (extStorages) {
+            // External storage paths must be identical on all the nodes (a cache has the single storage paths
+            // setting). Thus, a shared work directory is used, like in GridCommandHandlerListSnapshotTest.
+            cfg.setWorkDirectory(U.defaultWorkDirectory());
+
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                U.defaultWorkDirectory() + File.separator,
+                U.defaultWorkDirectory() + File.separator + "extStorage"
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", "extStorage");
+        }
+
+        cfg.setSnapshotThreadPoolSize(snpThrdPoolSz);
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        super.cleanPersistenceDir();
+
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTestsStarted() throws Exception {
+        super.beforeTestsStarted();
+
+        File workDir = new File(U.defaultWorkDirectory());
+
+        assertTrue(workDir.exists());
+
+        try {
+            Files.getPosixFilePermissions(workDir.toPath());
+
+            posixPermissions = true;
+        }
+        catch (UnsupportedOperationException ignored) {
+            // No-op.
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected <K, V> CacheConfiguration<K, V> txCacheConfig(CacheConfiguration<K, V> ccfg) {
+        // Speeds up the tests.
+        ccfg = super.txCacheConfig(ccfg)
+            .setAffinity(new RendezvousAffinityFunction(false, CACHE_PARTITIONS_COUNT))
+            .setBackups(1);
+
+        if (extStorages) {
+            assert !F.isEmpty(extStoragePaths);
+
+            ccfg.setStoragePaths(extStoragePaths);
+        }
+
+        return ccfg;
+    }
+
+    /** */
+    @Test
+    public void testConcurrentCreation() throws Exception {
+        // The test uses the thread blocking and conditional waiting. Won't proceed with 1 thread.
+        assumeTrue(snpThrdPoolSz > 1);
+
+        int grids = 3;
+        int testNodeIdx = 1;
+        int testNodeOrder = testNodeIdx + 1;
+
+        CountDownLatch beginSnpCreation = new CountDownLatch(grids);
+        CountDownLatch proceedSnpCreation = new CountDownLatch(1);
+
+        // Delays snapshot creation after its metadata is written.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public <M extends Serializable> void storeSnapshotMeta(M meta, File smf) {
+                            super.storeSnapshotMeta(meta, smf);
+
+                            beginSnpCreation.countDown();
+
+                            if (((IgniteEx)ctx.grid()).localNode().order() == testNodeOrder) {
+                                try {
+                                    assertTrue(proceedSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        IgniteFuture<Void> createSnpFut = snp(grid(0)).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary);
+
+        assertTrue(beginSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        SnapshotListTaskResult lstOpRes = listSnapshots(grid(0));
+
+        int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+        assertEquals(grids, snpsCnt);
+
+        SnapshotFileTree testSft = new SnapshotFileTree(grid(testNodeIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(SnapshotListTask.calculateDirectorySize(testSft.root()) > 0L);
+
+        proceedSnpCreation.countDown();
+
+        createSnpFut.get(getTestTimeout());
+    }
+
+    /** */
+    @Test
+    public void testCompletelyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(true);
+    }
+
+    /** */
+    @Test
+    public void testPartlyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(false);
+    }
+
+    /** */
+    private void doTestDeletionAfterMetaRead(boolean completeDeletion) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        CountDownLatch metaReadBeginLatch = new CountDownLatch(grids);
+        CountDownLatch metaReadProceedLatch = new CountDownLatch(1);
+
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft, boolean failIfCantRead) {
+                            metaReadBeginLatch.countDown();
+
+                            if (cctx.localNode().order() == testGridIdx + 1) {
+                                try {
+                                    assertTrue(metaReadProceedLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        long testSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+        assertTrue(testSnpSz > 0);
+
+        IgniteInternalFuture<SnapshotListTaskResult> lstOpFut = runAsync(() -> listSnapshots(grid(0)));
+
+        assertTrue(metaReadBeginLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        if (completeDeletion)
+            assertTrue(testSnpFt.root().exists() && U.delete(testSnpFt.root()) && !testSnpFt.root().exists());
+        else
+            assertTrue(testSnpFt.nodeStorage().exists() && U.delete(testSnpFt.nodeStorage()) && !testSnpFt.nodeStorage().exists());
+
+        metaReadProceedLatch.countDown();
+
+        SnapshotListTaskResult lstOpRes = lstOpFut.get();
+
+        if (completeDeletion) {
+            int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+            assertEquals(grids - 1, snpsCnt);
+        }
+        else {
+            int snpsCnt = 0;
+            boolean victimNodeFound = false;
+
+            for (int i = 0; i < lstOpRes.nodesIds().length; i++) {
+                UUID nid = lstOpRes.nodesIds()[i];
+
+                Map<String, SnapshotListJobResult.SnapshotInfo> nodeSnps = lstOpRes.nodesSnapshots()[i].snapshots();
+
+                snpsCnt += nodeSnps.size();
+
+                if (nid.equals(grid(testGridIdx).localNode().id())) {
+                    victimNodeFound = true;
+
+                    assertEquals(1, nodeSnps.size());
+
+                    long curSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+                    assertTrue(curSnpSz < testSnpSz);
+                }
+            }
+
+            assertTrue(victimNodeFound);
+            assertEquals(grids, snpsCnt);
+        }
+    }
+
+    /** */
+    @Test
+    public void testMissingSnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(true);
+    }
+
+    /** */
+    @Test
+    public void testEmptySnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(false);
+    }
+
+    /** */
+    private void doTestWrongSnapshotPath(boolean missing) throws Exception {
+        // Doesn't matter here, speeds up the tests.
+        assumeFalse(snpThrdPoolSz < 2);

Review Comment:
   Not sure we should skip the pool size parameter. We do use the pool. I've already met stuck issues in snapshots when a user configures just one thread.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324398996


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+            // Read local snapshots.
+            List<T2<SnapshotFileTree, Long>> locSnps = findLocalSnapshots(arg.src());
+
+            if (locSnps.isEmpty())
+                return new SnapshotListJobResult(Collections.emptyMap());
+
+            // Main sizes of the local snapshots (per snapshot name).
+            Map<String, Long> sizes = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional incremental parts descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> incs = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional external storages descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> extStors = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+
+            // Snapshot sizes, incremental parts and external storages futures.
+            List<Future<?>> futs = new ArrayList<>(locSnps.size() * 3);
+            // Names of snapshots with read failures.
+            Set<String> failedSnps = ConcurrentHashMap.newKeySet(locSnps.size() / 2);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            // Descriptions of the local snapshots: main sizes, external storages and incremental parts.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                SnapshotFileTree sft = snpPair.get1();
+                String snpName = sft.name();
+
+                // Future for the snapshot main size. It is the largest part of the job, thus it goes first.
+                Future<?> fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        sizes.put(snpName, calculateDirectorySize(sft.root()));
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to calculate snapshot's size, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional incremental parts.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo incDesc = incrementals(sft);
+
+                        if (incDesc != null)
+                            incs.put(snpName, incDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's incremental parts, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional external storages.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo extDesc = externalStorages(sft);
+
+                        if (extDesc != null)
+                            extStors.put(snpName, extDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's external storages, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+            }
+
+            // Wait for the futures.
+            for (Future<?> fut : futs) {
+                try {
+                    fut.get();
+                }
+                catch (ExecutionException e) {
+                    throw new IgniteException("Failed to read local node's snapshots.", e);
+                }
+                catch (InterruptedException e) {
+                    Thread.currentThread().interrupt();
+
+                    throw new IgniteException("Interrupted while reading local snapshots.", e);
+                }
+            }
+
+            // Result snapshot descriptions.
+            Map<String, SnapshotListJobResult.SnapshotInfo> resMap = new ConcurrentHashMap<>(locSnps.size() - failedSnps.size(), 1.0f);

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4358328792


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");
+
+            printer.accept("\tNode '%s' [uuid=%s]:".formatted(res.consistentIds()[nodeIdx], res.nodesIds()[nodeIdx]));
+
+            SnapshotListJobResult nodeResult = res.nodesSnapshots()[nodeIdx];
+
+            if (nodeResult.snapshots().isEmpty()) {
+                printer.accept("\t\t" + NO_SNAPSHOTS);
+
+                continue;
+            }
+
+            // Sorted snapshots names for better user reading.
+            new TreeMap<>(nodeResult.snapshots()).forEach((snpName, snpInfo) -> {
+                printer.accept("\t\tSnapshot '%s': totalSize=%s (%db), created='%s' (epoch=%d)".formatted(

Review Comment:
   Based on my research, the current format is close to well-known standards and common practices. Examples:
   
   - ISO 8601: '2026-09-18T12:00:00Z' / '2026-09-18T12:00:00+03:00'
   - DateTimeFormatter.ISO_INSTANT: '2024-01-15T14:30:45.123Z'
   - DateTimeFormatter.ISO_OFFSET_DATE_TIME: '2024-01-15T17:30:45.123+03:00'
   - Typical SQL-like timestamp: 'uuuu-MM-dd HH:mm:ss.SSS'
   - 
   The main principle is that larger units (such as years or months) come before smaller ones (such as hours, minutes, and seconds). The timezone (if required) comes last.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "zstan (via GitHub)" <gi...@apache.org>.
zstan commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4374837219


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.Comparator;
+import java.util.Date;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Consumer;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.management.SystemViewCommand;
+import org.apache.ignite.internal.management.SystemViewTask;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+import static org.apache.ignite.internal.management.SystemViewTask.SimpleType.DATE;
+import static org.apache.ignite.internal.management.SystemViewTask.SimpleType.NUMBER;
+import static org.apache.ignite.internal.management.SystemViewTask.SimpleType.STRING;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots were found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes, including their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        // Flag whether the node's result has at least one incremental snapshot.
+        AtomicBoolean hasIncrementals = new AtomicBoolean();
+        // Flag whether the node's result has at least one external storage.
+        AtomicBoolean hasExtStorages = new AtomicBoolean();

Review Comment:
   u not need to use Atomic here, not so good but more accurate is to use T2 or custom structure, plz fix it



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+            // Read local snapshots.
+            List<T2<SnapshotFileTree, Long>> locSnps = findLocalSnapshots(arg.src());
+
+            if (locSnps.isEmpty())
+                return new SnapshotListJobResult(Collections.emptyMap());
+
+            // Main sizes of the local snapshots (per snapshot name).
+            Map<String, Long> sizes = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional incremental parts descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> incs = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional external storages descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> extStors = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+
+            // Snapshot sizes, incremental parts and external storages futures.
+            List<Future<?>> futs = new ArrayList<>(locSnps.size() * 3);
+            // Names of snapshots with read failures.
+            Set<String> failedSnps = ConcurrentHashMap.newKeySet(locSnps.size() / 2);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            // Descriptions of the local snapshots: main sizes, external storages and incremental parts.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                SnapshotFileTree sft = snpPair.get1();
+                String snpName = sft.name();
+
+                // Future for the snapshot main size. It is the largest part of the job, thus it goes first.
+                Future<?> fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        sizes.put(snpName, calculateDirectorySize(sft.root()));
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to calculate snapshot's size, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional incremental parts.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo incDesc = incrementals(sft);
+
+                        if (incDesc != null)
+                            incs.put(snpName, incDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's incremental parts, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional external storages.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo extDesc = externalStorages(sft);
+
+                        if (extDesc != null)
+                            extStors.put(snpName, extDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's external storages, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+            }
+
+            // Wait for the futures.
+            for (Future<?> fut : futs) {
+                try {
+                    fut.get();
+                }
+                catch (ExecutionException e) {
+                    throw new IgniteException("Failed to read local node's snapshots.", e);
+                }
+                catch (InterruptedException e) {
+                    Thread.currentThread().interrupt();
+
+                    throw new IgniteException("Interrupted while reading local snapshots.", e);
+                }
+            }
+
+            // Result snapshot descriptions.
+            Map<String, SnapshotListJobResult.SnapshotInfo> resMap = new ConcurrentHashMap<>(locSnps.size() - failedSnps.size(), 1.0f);
+
+            // Reduce results.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                String snpName = snpPair.get1().name();
+
+                if (failedSnps.contains(snpName))
+                    continue;
+
+                Long size = sizes.get(snpName);
+
+                // The size must be calculated unless the snapshot has failed.
+                assert size != null;
+
+                SnapshotListJobResult.SnapshotInfo snpDesc = new SnapshotListJobResult.SnapshotInfo(
+                    size,
+                    snpPair.get2(),
+                    extStors.get(snpName),
+                    incs.get(snpName)
+                );
+
+                resMap.put(snpName, snpDesc);
+            }
+
+            return new SnapshotListJobResult(resMap);
+        }
+
+        /** */
+        private @Nullable SnapshotListJobResult.SnapshotInfo externalStorages(SnapshotFileTree sft) {
+            int extStoragesCnt = 0;
+            long extStoragesSize = 0;
+
+            for (File extraStorage : sft.allStorages().toList()) {
+                if (sft.nodeStorage().equals(extraStorage))
+                    continue;
+
+                try {
+                    extStoragesSize += calculateDirectorySize(extraStorage);
+                }
+                catch (Exception e) {
+                    log.warning("Failed to calculate snapshot's external storage size, storage ignored [extraStorage='" +
+                        extraStorage + "']", e);
+
+                    continue;
+                }
+
+                extStoragesCnt++;
+            }
+
+            return extStoragesCnt == 0 ? null : new SnapshotListJobResult.SnapshotInfo(extStoragesCnt, extStoragesSize);
+        }
+
+        /** @return Snapshot file tree and creation time from the snapshot metadata. */
+        private List<T2<SnapshotFileTree, Long>> findLocalSnapshots(@Nullable String snpPath) {
+            // The tree is used only to extract the snapshots root directory. Such snapshot name is actually unallowed.
+            File[] dirsToParse = new SnapshotFileTree(ignite.context(), "snp", snpPath).root().getParentFile().listFiles();

Review Comment:
   Seems you can replace this strange code for (a bit clear) : 
   `org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree#root`
   wdyt ?



##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListTest.java:
##########
@@ -0,0 +1,918 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.io.Serializable;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Stream;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.io.GridFileUtils;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.lang.IgniteFuture;
+import org.apache.ignite.plugin.AbstractTestPluginProvider;
+import org.apache.ignite.plugin.PluginConfiguration;
+import org.apache.ignite.plugin.PluginContext;
+import org.apache.ignite.plugin.PluginProvider;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runners.Parameterized;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.configuration.IgniteConfiguration.DFLT_SNAPSHOT_THREAD_POOL_SIZE;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.cartesianProduct;
+import static org.apache.ignite.testframework.GridTestUtils.runAsync;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Cluster-wide snapshot list procedure tests. */
+public class IgniteClusterSnapshotListTest extends AbstractSnapshotSelfTest {
+    /** Number of cache keys to pre-create at node startup. */
+    private static final int CACHE_KEYS_RANGE = 15;
+
+    /** Number of partitions of the test cache. */
+    private static final int CACHE_PARTITIONS_COUNT = 4;
+
+    /** Flag showing that the filesystem supports POSIX permissions. */
+    private static boolean posixPermissions;
+
+    /** Size of the snapshot utility thread pool. */
+    @Parameterized.Parameter(2)
+    public int snpThrdPoolSz;
+
+    /** */
+    private PluginProvider<PluginConfiguration> pluginProvider;
+
+    /** Flag to spread the test cache data over external storages. */
+    private boolean extStorages;
+
+    /** Resolved external storage paths. {@code null} if {@link #extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Parameters. */
+    @Parameterized.Parameters(name = "encryption={0}, onlyPrimary={1}, snpThrdPoolSz={2}")
+    public static Collection<Object[]> params() {
+        return cartesianProduct(
+            encryptionParameters(), // Encryption.
+            F.asList(false, true), // Only primary.
+            F.asList(DFLT_SNAPSHOT_THREAD_POOL_SIZE, 1) // Snapshots thread pool size.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName)
+            .setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        if (pluginProvider != null)
+            cfg.setPluginProviders(pluginProvider);
+
+        if (extStorages) {
+            // External storage paths must be identical on all nodes (a cache has a single storage path setting).
+            // Thus, a shared work directory is used, like in GridCommandHandlerListSnapshotTest.
+            cfg.setWorkDirectory(U.defaultWorkDirectory());
+
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                U.defaultWorkDirectory() + File.separator,
+                U.defaultWorkDirectory() + File.separator + "extStorage"
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", "extStorage");
+        }
+
+        cfg.setSnapshotThreadPoolSize(snpThrdPoolSz);
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        super.cleanPersistenceDir();
+
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTestsStarted() throws Exception {
+        super.beforeTestsStarted();
+
+        File workDir = new File(U.defaultWorkDirectory());
+
+        assertTrue(workDir.exists());
+
+        try {
+            Files.getPosixFilePermissions(workDir.toPath());
+
+            posixPermissions = true;
+        }
+        catch (UnsupportedOperationException ignored) {
+            // No-op.
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected <K, V> CacheConfiguration<K, V> txCacheConfig(CacheConfiguration<K, V> ccfg) {
+        // Speeds up the tests.
+        ccfg = super.txCacheConfig(ccfg)
+            .setAffinity(new RendezvousAffinityFunction(false, CACHE_PARTITIONS_COUNT))
+            .setBackups(1);
+
+        if (extStorages) {
+            assert !F.isEmpty(extStoragePaths);
+
+            ccfg.setStoragePaths(extStoragePaths);
+        }
+
+        return ccfg;
+    }
+
+    /** */
+    @Test
+    public void testConcurrentCreation() throws Exception {
+        // The test relies on thread blocking and conditional waiting, so it does not run with a single thread.
+        assumeTrue(snpThrdPoolSz > 1);
+
+        int grids = 3;
+        int testNodeIdx = 1;
+        int testNodeOrder = testNodeIdx + 1;
+
+        CountDownLatch beginSnpCreation = new CountDownLatch(grids);
+        CountDownLatch proceedSnpCreation = new CountDownLatch(1);
+
+        // Delays snapshot creation after its metadata is written.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public <M extends Serializable> void storeSnapshotMeta(M meta, File smf) {
+                            super.storeSnapshotMeta(meta, smf);
+
+                            beginSnpCreation.countDown();
+
+                            if (((IgniteEx)ctx.grid()).localNode().order() == testNodeOrder) {
+                                try {
+                                    assertTrue(proceedSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        IgniteFuture<Void> createSnpFut = snp(grid(0)).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary);
+
+        assertTrue(beginSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        SnapshotListTaskResult lstOpRes = listSnapshots(grid(0));
+
+        int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+        assertEquals(grids, snpsCnt);
+
+        SnapshotFileTree testSft = new SnapshotFileTree(grid(testNodeIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(GridFileUtils.sizeOfDirectory(testSft.root()) > 0L);
+
+        proceedSnpCreation.countDown();
+
+        createSnpFut.get(getTestTimeout());
+    }
+
+    /** */
+    @Test
+    public void testCompletelyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(true);
+    }
+
+    /** */
+    @Test
+    public void testPartlyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(false);
+    }
+
+    /** */
+    private void doTestDeletionAfterMetaRead(boolean completeDeletion) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        CountDownLatch metaReadBeginLatch = new CountDownLatch(grids);
+        CountDownLatch metaReadProceedLatch = new CountDownLatch(1);
+
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft, boolean failIfCantRead) {
+                            metaReadBeginLatch.countDown();
+
+                            if (cctx.localNode().order() == testGridIdx + 1) {
+                                try {
+                                    assertTrue(metaReadProceedLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        long testSnpSz = GridFileUtils.sizeOfDirectory(testSnpFt.root());
+
+        assertTrue(testSnpSz > 0);
+
+        IgniteInternalFuture<SnapshotListTaskResult> lstOpFut = runAsync(() -> listSnapshots(grid(0)));
+
+        assertTrue(metaReadBeginLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        if (completeDeletion)
+            assertTrue(testSnpFt.root().exists() && U.delete(testSnpFt.root()) && !testSnpFt.root().exists());
+        else
+            assertTrue(testSnpFt.nodeStorage().exists() && U.delete(testSnpFt.nodeStorage()) && !testSnpFt.nodeStorage().exists());
+
+        metaReadProceedLatch.countDown();
+
+        SnapshotListTaskResult lstOpRes = lstOpFut.get();
+
+        if (completeDeletion) {
+            int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+            assertEquals(grids - 1, snpsCnt);
+        }
+        else {
+            int snpsCnt = 0;
+            boolean victimNodeFound = false;
+
+            for (int i = 0; i < lstOpRes.nodesIds().length; i++) {
+                UUID nid = lstOpRes.nodesIds()[i];
+
+                Map<String, SnapshotListJobResult.SnapshotInfo> nodeSnps = lstOpRes.nodesSnapshots()[i].snapshots();
+
+                snpsCnt += nodeSnps.size();
+
+                if (nid.equals(grid(testGridIdx).localNode().id())) {
+                    victimNodeFound = true;
+
+                    assertEquals(1, nodeSnps.size());
+
+                    long curSnpSz = GridFileUtils.sizeOfDirectory(testSnpFt.root());
+
+                    assertTrue(curSnpSz < testSnpSz);
+                }
+            }
+
+            assertTrue(victimNodeFound);
+            assertEquals(grids, snpsCnt);
+        }
+    }
+
+    /** */
+    @Test
+    public void testEmptySnapshotPath() throws Exception {
+        startGrids(3);
+
+        File path = new File(U.defaultWorkDirectory(), "not_snapshots");
+
+        SnapshotListJobResult[] listOperationRes = listSnapshots(grid(0), path.getAbsolutePath()).nodesSnapshots();
+
+        int cnt = Stream.of(listOperationRes).mapToInt(nodeRes -> nodeRes.snapshots().size()).sum();
+
+        assertEquals(0, cnt);
+    }
+
+    /** */
+    @Test
+    public void testDefaultSnapshotPath() throws Exception {
+        doTestSnapshotPath(null);
+    }
+
+    /** */
+    @Test
+    @Ignore("https://issues.apache.org/jira/browse/IGNITE-29126")
+    public void testRelativeSnapshotPath() throws Exception {
+        doTestSnapshotPath("ex_snapshots");
+    }
+
+    /** */
+    @Test
+    public void testAbsoluteSnapshotPath() throws Exception {
+        doTestSnapshotPath(new File(U.defaultWorkDirectory(), "ex_snapshots").getAbsolutePath());
+    }
+
+    /** */
+    private void doTestSnapshotPath(@Nullable String path) throws Exception {
+        int grids = 3;
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        snp(grid(0)).createSnapshot(SNAPSHOT_NAME, path, false, onlyPrimary).get(getTestTimeout());
+
+        SnapshotListJobResult[] lstOpRes = listSnapshots(grid(0), path).nodesSnapshots();
+
+        int cnt = Stream.of(lstOpRes).mapToInt(nodeRes -> nodeRes.snapshots().size()).sum();
+
+        assertEquals(grids, cnt);
+    }
+
+    /** */
+    @Test
+    public void testMissingMeta() throws Exception {
+        doTestWithWrongMeta(false);
+    }
+
+    /** */
+    @Test
+    public void testCorruptedMeta() throws Exception {
+        doTestWithWrongMeta(true);
+    }
+
+    /**
+     * Tests snapshot lists when one snapshot metadata cannot be read.
+     *
+     * @param corruptFile If {@code true}, corrupts metadata. Otherwise, deletes metadata.
+     */
+    private void doTestWithWrongMeta(boolean corruptFile) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(testSnpFt.meta().exists());
+
+        if (corruptFile) {
+            try (RandomAccessFile raf = new RandomAccessFile(testSnpFt.meta(), "rw")) {
+                raf.write(UUID.randomUUID().toString().getBytes());
+            }
+        }
+        else
+            assertTrue(testSnpFt.meta().delete() && !testSnpFt.meta().exists());
+
+        SnapshotListTaskResult res = listSnapshots(grid(0));
+
+        int foundSnpsCnt = 0;
+        boolean victimNodeFound = false;
+
+        for (int i = 0; i < res.nodesIds().length; i++) {
+            UUID nid = res.nodesIds()[i];
+
+            foundSnpsCnt += res.nodesSnapshots()[i].snapshots().size();
+
+            if (nid.equals(grid(testGridIdx).localNode().id())) {
+                victimNodeFound = true;
+
+                assertTrue(res.nodesSnapshots()[i].snapshots().isEmpty());
+            }
+        }
+
+        assertTrue(victimNodeFound);
+        assertEquals(grids - 1, foundSnpsCnt);
+    }
+
+    /** */
+    @Test
+    public void testMissingIncrementalMeta() throws Exception {
+        doTestWithWrongIncrementalMeta(false);
+    }
+
+    /** */
+    @Test
+    public void testCorruptedIncrementalMeta() throws Exception {
+        doTestWithWrongIncrementalMeta(true);
+    }
+
+    /**
+     * Tests snapshot list when incremental snapshot metadata cannot be read.
+     * The main snapshot should still be listed, but without incremental info on the affected node.
+     *
+     * @param corruptFile If {@code true}, corrupts metadata. Otherwise, deletes metadata.
+     */
+    private void doTestWithWrongIncrementalMeta(boolean corruptFile) throws Exception {
+        // Incremental snapshots do not support the only-primary mode or encryption.
+        assumeFalse(onlyPrimary || encryption);
+
+        int grids = 3;
+        int testGridIdx = 1;
+        int incsCnt = 3;
+
+        IgniteEx ig = startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        snp(ig).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary).get(getTestTimeout());
+
+        for (int i = 0; i < incsCnt; ++i) {
+            try (IgniteDataStreamer<Integer, Integer> ds = grid(0).dataStreamer(DEFAULT_CACHE_NAME)) {
+                for (int kv = (i + 1) * CACHE_KEYS_RANGE; kv < (i + 1) * CACHE_KEYS_RANGE * 2; ++kv)
+                    ds.addData(kv, kv);
+            }
+
+            snp(ig).createSnapshot(SNAPSHOT_NAME, null, true, onlyPrimary).get(getTestTimeout());
+        }
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+        SnapshotFileTree.IncrementalSnapshotFileTree incFt = testSnpFt.incrementalSnapshotFileTree(1);
+        File incMeta = incFt.meta();
+
+        assertTrue(incMeta.exists());
+
+        if (corruptFile) {
+            try (RandomAccessFile raf = new RandomAccessFile(incMeta, "rw")) {
+                raf.write(UUID.randomUUID().toString().getBytes());
+            }
+        }
+        else
+            assertTrue(incMeta.delete() && !incMeta.exists());
+
+        SnapshotListTaskResult res = listSnapshots(grid(0));
+
+        for (int i = 0; i < res.nodesIds().length; i++) {
+            UUID nid = res.nodesIds()[i];
+
+            Map<String, SnapshotListJobResult.SnapshotInfo> snps = res.nodesSnapshots()[i].snapshots();
+
+            assertEquals(1, snps.size());
+
+            SnapshotListJobResult.SnapshotInfo info = snps.get(SNAPSHOT_NAME);
+
+            assertNotNull(info);
+            assertNotNull(info.incrementals());
+
+            if (nid.equals(grid(testGridIdx).localNode().id()))
+                assertEquals(incsCnt - 1, info.incrementals().number().intValue());
+            else
+                assertEquals(incsCnt, info.incrementals().number().intValue());
+        }
+    }
+
+    /**
+     * Test snapshot list operation when a node can't read some snapshot part due to insufficient permissions.
+     * That is, a test node can read the snapshot metadata but cannot read some of the snapshot data.
+     */
+    @Test
+    public void testDeniedPermissions() throws Exception {
+        assumeTrue(posixPermissions);
+        // We rely on the snapshot sizes here. Better to avoid empty data nodes not to become flaky.
+        assumeFalse(onlyPrimary);
+
+        int grids = 3;
+        int testGridIdx = 1;
+
+        // Permissions to restore.
+        Map<Path, Set<PosixFilePermission>> oldPerms = new ConcurrentHashMap<>();
+        // The 'change permissions' flag.
+        AtomicBoolean changePermissions = new AtomicBoolean(true);
+
+        // Deny reading the whole snapshot part of one node.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft,
+                            boolean failIfCantRead) {
+                            if (changePermissions.get() && ctx.localNode().order() == testGridIdx + 1) {
+                                File victimDir = sft.nodeStorage();
+
+                                assertTrue(victimDir.exists());
+                                assertTrue(victimDir.isDirectory());
+
+                                // Ensure the blocked snapshot part has a non-zero size; the sizes are compared later.
+                                try {
+                                    assertTrue(GridFileUtils.sizeOfDirectory(victimDir) > 0L);
+                                }
+                                catch (IOException e) {
+                                    throw new IllegalStateException(e);
+                                }
+
+                                Path victimDirPath = victimDir.toPath();
+
+                                try {
+                                    Set<PosixFilePermission> perms = Files.getPosixFilePermissions(victimDirPath);
+
+                                    assertFalse(perms.isEmpty());
+
+                                    // Deny reading.
+                                    Files.setPosixFilePermissions(victimDirPath, PosixFilePermissions.fromString("---------"));
+
+                                    // Save actual permissions to restore.
+                                    oldPerms.put(victimDirPath, perms);
+                                }
+                                catch (Exception e) {
+                                    throw new IgniteException("Unable to set the test POSIX permissions.", e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, true);
+
+        UUID testNodeId = grid(testGridIdx).localNode().id();
+
+        // Snapshot size with restricted permissions.
+        long testSize0 = 0L;
+        // Snapshot size with normal permissions.
+        long testSize1 = 0L;
+
+        SnapshotListTaskResult listTaskRes;
+
+        try {
+            // First run.
+            listTaskRes = listSnapshots(grid(0));
+
+            assertFalse(oldPerms.isEmpty());
+
+            for (int i = 0; i < listTaskRes.nodesIds().length; i++) {
+                UUID nodeId = listTaskRes.nodesIds()[i];
+
+                // Store the partly read snapshot size.
+                if (nodeId.equals(testNodeId)) {
+                    Map<String, SnapshotListJobResult.SnapshotInfo> snps = listTaskRes.nodesSnapshots()[i].snapshots();
+
+                    assertEquals(1, snps.size());
+
+                    testSize0 = snps.get(SNAPSHOT_NAME).size();
+                }
+            }
+
+            // Ensure that we've found and read the snapshot on the test node.
+            assertTrue(testSize0 > 0L);
+        }
+        finally {
+            // Restore the permissions in any case.
+            oldPerms.forEach((path, perms) -> {
+                try {
+                    Files.setPosixFilePermissions(path, perms);
+                }
+                catch (IOException e) {
+                    throw new IllegalStateException(e);
+                }
+            });
+        }
+
+        // Relaunch the operation.
+        changePermissions.set(false);
+
+        listTaskRes = listSnapshots(grid(0));
+
+        for (int i = 0; i < listTaskRes.nodesIds().length; i++) {
+            UUID nodeId = listTaskRes.nodesIds()[i];
+
+            // Store the fully read snapshot size.
+            if (nodeId.equals(testNodeId)) {
+                Map<String, SnapshotListJobResult.SnapshotInfo> snps = listTaskRes.nodesSnapshots()[i].snapshots();
+
+                assertEquals(1, snps.size());
+
+                testSize1 = snps.get(SNAPSHOT_NAME).size();
+            }
+        }
+
+        // Ensure that the newly calculated size is greater than in the previous run.
+        assertTrue(testSize1 > 0L);
+        assertTrue(testSize1 > testSize0);
+    }
+
+    /** */
+    @Test
+    public void testSnapshotListsDates() throws Exception {
+        // Incremental snapshots do not support the only-primary mode or encryption.
+        assumeFalse(onlyPrimary || encryption);
+
+        int grids = 3;
+
+        IgniteEx ig = startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        long time0 = U.currentTimeMillis();
+
+        // Wait so that the timestamps differ.
+        U.sleep(300L);

Review Comment:
   do we really want to chech for strong ">" , I mean : `assertTrue(info.date() > time0);` I just don\`t like to use sleep in tests without strong needs, probably you remove it and near too and change for : assertTrue(info.date() >= time0);
   also time0 - non informative and need to be renamed



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',

Review Comment:
   I\`d prefer to fix it, but if other places are uses the same approach - it\`s up to you



##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListRollingUpgradeTest.java:
##########
@@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.UUID;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.CacheAtomicityMode;
+import org.apache.ignite.cache.CacheMode;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.DataRegionConfiguration;
+import org.apache.ignite.configuration.DataStorageConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.rollingupgrade.AbstractRollingUpgradeTest;
+import org.apache.ignite.internal.util.distributed.SingleNodeMessage;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.junit.Test;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.internal.TestRecordingCommunicationSpi.spi;
+import static org.apache.ignite.internal.util.distributed.DistributedProcess.DistributedProcessType.RU_PREPARE_VERSION_FINALIZATION;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+
+/** */
+public class IgniteClusterSnapshotListRollingUpgradeTest extends AbstractRollingUpgradeTest {
+    /** */
+    private static final int ALL_GRIDS = 4;
+
+    /** */
+    private static final int CLIENTS = 1;
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        super.afterTest();
+
+        // Clean everything, including the separated working directories.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName, String ver) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName, ver);
+
+        DataRegionConfiguration dataRegionCfg = new DataRegionConfiguration()
+            .setPersistenceEnabled(true)
+            .setMaxSize(DataStorageConfiguration.DFLT_DATA_REGION_INITIAL_SIZE);
+
+        cfg.setDataStorageConfiguration(new DataStorageConfiguration().setDefaultDataRegionConfiguration(dataRegionCfg));
+
+        cfg.setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        return cfg;
+    }
+
+    /** */
+    @Test
+    public void testParallelRollingUpgradeInProgress() throws Exception {
+        for (int i = 0; i < ALL_GRIDS; i++)
+            startGrid(i, "2.18.0", i >= ALL_GRIDS - CLIENTS);
+
+        grid(0).cluster().active(true);
+
+        int testNodeIx = ALL_GRIDS - CLIENTS - 1;
+
+        createCacheAndSnapshot(testNodeIx);
+
+        ru(grid(testNodeIx)).enableVersionUpgrade();
+
+        for (int i = 0; i < ALL_GRIDS; i++) {
+            assertTrue(ru(grid(i)).isVersionUpgradeEnabled());
+
+            upgradeNodeVersion(i, "2.18.0", MASTER_VER_REF);
+        }
+
+        spi(grid(testNodeIx)).blockMessages((node, msg) -> msg instanceof SingleNodeMessage<?> snm &&
+            snm.type() == RU_PREPARE_VERSION_FINALIZATION.ordinal());
+
+        IgniteInternalFuture<?> finalizeFut = GridTestUtils.runAsync(() -> ru(testNodeIx).finalizeClusterVersion());
+
+        assertTrue(spi(grid(testNodeIx)).waitForBlocked(1, getTestTimeout()));
+
+        ensureSnapshotListCommandFails();
+
+        spi(grid(testNodeIx)).stopBlock();
+
+        assertFalse(spi(grid(testNodeIx)).hasBlockedMessages());
+
+        finalizeFut.get(getTestTimeout());
+
+        for (int i = 0; i < ALL_GRIDS; i++) {
+            int i0 = i;
+
+            assertTrue(waitForCondition(() -> !ru(grid(i0)).isVersionUpgradeEnabled(), getTestTimeout()));
+        }
+
+        ensureSnapshotListSucceeds();
+    }
+
+    /** */
+    @Test
+    public void testSnapshotListFeature() throws Exception {
+        for (int i = 0; i < ALL_GRIDS; i++)
+            startGrid(i, "2.18.0", i >= ALL_GRIDS - CLIENTS);
+
+        grid(0).cluster().active(true);
+
+        createCacheAndSnapshot(1);
+
+        ensureSnapshotListCommandFails();
+
+        ru(grid(0)).enableVersionUpgrade();
+
+        for (int i = 0; i < ALL_GRIDS; i++) {
+            assertTrue(ru(grid(i)).isVersionUpgradeEnabled());
+
+            upgradeNodeVersion(i, "2.18.0", MASTER_VER_REF);
+
+            ensureSnapshotListCommandFails();
+        }
+
+        ru(grid(1)).finalizeClusterVersion();
+
+        for (int i = 0; i < ALL_GRIDS; i++) {
+            int i0 = i;
+
+            assertTrue(waitForCondition(() -> !ru(grid(i0)).isVersionUpgradeEnabled(), getTestTimeout()));
+        }
+
+        ensureSnapshotListSucceeds();
+    }
+
+    /** */
+    private void ensureSnapshotListSucceeds() throws Exception {
+        for (int i = 0; i < ALL_GRIDS; i++) {
+            for (int k = 0; k < ALL_GRIDS; k++) {
+                int i0 = i;
+                int k0 = k;

Review Comment:
   redundant variables



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java:
##########
@@ -0,0 +1,140 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.util.Map;
+import org.apache.ignite.internal.Order;
+import org.apache.ignite.internal.dto.IgniteDataTransferObject;
+import org.jetbrains.annotations.Nullable;
+
+/** Per-node result of the snapshot list task. Contains information about the snapshots found on the node. */
+public final class SnapshotListJobResult extends IgniteDataTransferObject {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** Local node's snapshot descriptions. */
+    @Order(0)
+    Map<String, SnapshotInfo> snapshots;
+
+    /** Default constructor for serialization purposes. */
+    public SnapshotListJobResult() {
+        // No-op.
+    }
+
+    /** */
+    public SnapshotListJobResult(Map<String, SnapshotInfo> snapshots) {
+        this.snapshots = snapshots;
+    }
+
+    /** @return Snapshot descriptions by snapshot name. */
+    public Map<String, SnapshotInfo> snapshots() {
+        return snapshots;
+    }
+
+    /** Holds information about a snapshot or its parts: size, creation time, number of incremental parts or external storages. */
+    public static class SnapshotInfo extends IgniteDataTransferObject {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** Total size of a snapshot. Or size of its external storages or incremental parts. */
+        @Order(0)
+        long size;
+
+        /** Creation date of the snapshot or of its incremental parts. */
+        @Order(1)
+        @Nullable Long date;

Review Comment:
   unboxing on constructor and boxing here, why do we need it ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324651099


##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListTest.java:
##########
@@ -0,0 +1,936 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.io.Serializable;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Stream;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.lang.IgniteFuture;
+import org.apache.ignite.plugin.AbstractTestPluginProvider;
+import org.apache.ignite.plugin.PluginConfiguration;
+import org.apache.ignite.plugin.PluginContext;
+import org.apache.ignite.plugin.PluginProvider;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runners.Parameterized;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.configuration.IgniteConfiguration.DFLT_SNAPSHOT_THREAD_POOL_SIZE;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.cartesianProduct;
+import static org.apache.ignite.testframework.GridTestUtils.runAsync;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Cluster-wide snapshot list procedure tests. */
+public class IgniteClusterSnapshotListTest extends AbstractSnapshotSelfTest {
+    /** Number of cache keys to pre-create at node startup. */
+    private static final int CACHE_KEYS_RANGE = 15;
+
+    /** Number of partitions of the test cache. */
+    private static final int CACHE_PARTITIONS_COUNT = 4;
+
+    /** Flag showing that the filesystem supports POSIX permissions. */
+    private static boolean posixPermissions;
+
+    /** Size of the snapshot utility thread pool. */
+    @Parameterized.Parameter(2)
+    public int snpThrdPoolSz;
+
+    /** */
+    private PluginProvider<PluginConfiguration> pluginProvider;
+
+    /** Flag to spread the test cache data over the external storages. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@link #extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Parameters. */
+    @Parameterized.Parameters(name = "encryption={0}, onlyPrimary={1}, snpThrdPoolSz={2}")
+    public static Collection<Object[]> params() {
+        return cartesianProduct(
+            encryptionParameters(), // Encryption.
+            F.asList(false, true), // Only primary.
+            F.asList(DFLT_SNAPSHOT_THREAD_POOL_SIZE, 1) // Snapshots thread pool size.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName)
+            .setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        if (pluginProvider != null)
+            cfg.setPluginProviders(pluginProvider);
+
+        if (extStorages) {
+            // External storage paths must be identical on all the nodes (a cache has the single storage paths
+            // setting). Thus, a shared work directory is used, like in GridCommandHandlerListSnapshotTest.
+            cfg.setWorkDirectory(U.defaultWorkDirectory());
+
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                U.defaultWorkDirectory() + File.separator,
+                U.defaultWorkDirectory() + File.separator + "extStorage"
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", "extStorage");
+        }
+
+        cfg.setSnapshotThreadPoolSize(snpThrdPoolSz);
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        super.cleanPersistenceDir();
+
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTestsStarted() throws Exception {
+        super.beforeTestsStarted();
+
+        File workDir = new File(U.defaultWorkDirectory());
+
+        assertTrue(workDir.exists());
+
+        try {
+            Files.getPosixFilePermissions(workDir.toPath());
+
+            posixPermissions = true;
+        }
+        catch (UnsupportedOperationException ignored) {
+            // No-op.
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected <K, V> CacheConfiguration<K, V> txCacheConfig(CacheConfiguration<K, V> ccfg) {
+        // Speeds up the tests.
+        ccfg = super.txCacheConfig(ccfg)
+            .setAffinity(new RendezvousAffinityFunction(false, CACHE_PARTITIONS_COUNT))
+            .setBackups(1);
+
+        if (extStorages) {
+            assert !F.isEmpty(extStoragePaths);
+
+            ccfg.setStoragePaths(extStoragePaths);
+        }
+
+        return ccfg;
+    }
+
+    /** */
+    @Test
+    public void testConcurrentCreation() throws Exception {
+        // The test uses the thread blocking and conditional waiting. Won't proceed with 1 thread.
+        assumeTrue(snpThrdPoolSz > 1);
+
+        int grids = 3;
+        int testNodeIdx = 1;
+        int testNodeOrder = testNodeIdx + 1;
+
+        CountDownLatch beginSnpCreation = new CountDownLatch(grids);
+        CountDownLatch proceedSnpCreation = new CountDownLatch(1);
+
+        // Delays snapshot creation after its metadata is written.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public <M extends Serializable> void storeSnapshotMeta(M meta, File smf) {
+                            super.storeSnapshotMeta(meta, smf);
+
+                            beginSnpCreation.countDown();
+
+                            if (((IgniteEx)ctx.grid()).localNode().order() == testNodeOrder) {
+                                try {
+                                    assertTrue(proceedSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        IgniteFuture<Void> createSnpFut = snp(grid(0)).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary);
+
+        assertTrue(beginSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        SnapshotListTaskResult lstOpRes = listSnapshots(grid(0));
+
+        int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+        assertEquals(grids, snpsCnt);
+
+        SnapshotFileTree testSft = new SnapshotFileTree(grid(testNodeIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(SnapshotListTask.calculateDirectorySize(testSft.root()) > 0L);
+
+        proceedSnpCreation.countDown();
+
+        createSnpFut.get(getTestTimeout());
+    }
+
+    /** */
+    @Test
+    public void testCompletelyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(true);
+    }
+
+    /** */
+    @Test
+    public void testPartlyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(false);
+    }
+
+    /** */
+    private void doTestDeletionAfterMetaRead(boolean completeDeletion) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        CountDownLatch metaReadBeginLatch = new CountDownLatch(grids);
+        CountDownLatch metaReadProceedLatch = new CountDownLatch(1);
+
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft, boolean failIfCantRead) {
+                            metaReadBeginLatch.countDown();
+
+                            if (cctx.localNode().order() == testGridIdx + 1) {
+                                try {
+                                    assertTrue(metaReadProceedLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        long testSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+        assertTrue(testSnpSz > 0);
+
+        IgniteInternalFuture<SnapshotListTaskResult> lstOpFut = runAsync(() -> listSnapshots(grid(0)));
+
+        assertTrue(metaReadBeginLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        if (completeDeletion)
+            assertTrue(testSnpFt.root().exists() && U.delete(testSnpFt.root()) && !testSnpFt.root().exists());
+        else
+            assertTrue(testSnpFt.nodeStorage().exists() && U.delete(testSnpFt.nodeStorage()) && !testSnpFt.nodeStorage().exists());
+
+        metaReadProceedLatch.countDown();
+
+        SnapshotListTaskResult lstOpRes = lstOpFut.get();
+
+        if (completeDeletion) {
+            int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+            assertEquals(grids - 1, snpsCnt);
+        }
+        else {
+            int snpsCnt = 0;
+            boolean victimNodeFound = false;
+
+            for (int i = 0; i < lstOpRes.nodesIds().length; i++) {
+                UUID nid = lstOpRes.nodesIds()[i];
+
+                Map<String, SnapshotListJobResult.SnapshotInfo> nodeSnps = lstOpRes.nodesSnapshots()[i].snapshots();
+
+                snpsCnt += nodeSnps.size();
+
+                if (nid.equals(grid(testGridIdx).localNode().id())) {
+                    victimNodeFound = true;
+
+                    assertEquals(1, nodeSnps.size());
+
+                    long curSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+                    assertTrue(curSnpSz < testSnpSz);
+                }
+            }
+
+            assertTrue(victimNodeFound);
+            assertEquals(grids, snpsCnt);
+        }
+    }
+
+    /** */
+    @Test
+    public void testMissingSnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(true);
+    }
+
+    /** */
+    @Test
+    public void testEmptySnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(false);
+    }
+
+    /** */
+    private void doTestWrongSnapshotPath(boolean missing) throws Exception {
+        // Doesn't matter here, speeds up the tests.
+        assumeFalse(snpThrdPoolSz < 2);
+
+        doTestSnapshotPath(null);

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324682968


##########
modules/core/src/test/resources/org.apache.ignite.util/GridCommandHandlerClusterByClassWithSSLTest_help.output:
##########
@@ -269,6 +269,12 @@ This utility can do the following commands:
   Get the status of the current snapshot operation:
     control.(sh|bat) --snapshot status
 
+  Lists all snapshots on all online server nodes with their sizes:

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6417305978

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotCheckCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotDeleteCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotRestoreCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4363457008


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");

Review Comment:
   Done. The table format applied.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6250099859

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsSingleMessage.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6251329095

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsSingleMessage.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "zstan (via GitHub)" <gi...@apache.org>.
zstan commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4345725004


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");
+
+            printer.accept("\tNode '%s' [uuid=%s]:".formatted(res.consistentIds()[nodeIdx], res.nodesIds()[nodeIdx]));
+
+            SnapshotListJobResult nodeResult = res.nodesSnapshots()[nodeIdx];
+
+            if (nodeResult.snapshots().isEmpty()) {
+                printer.accept("\t\t" + NO_SNAPSHOTS);
+
+                continue;
+            }
+
+            // Sorted snapshots names for better user reading.
+            new TreeMap<>(nodeResult.snapshots()).forEach((snpName, snpInfo) -> {
+                printer.accept("\t\tSnapshot '%s': totalSize=%s (%db), created='%s' (epoch=%d)".formatted(

Review Comment:
   Also seems we need table with columns format here, if it common case for control utility output
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6085026303

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "zstan (via GitHub)" <gi...@apache.org>.
zstan commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4311979330


##########
modules/control-utility/src/test/java/org/apache/ignite/util/GridCommandHandlerListSnapshotTest.java:
##########
@@ -0,0 +1,348 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.util;
+
+import java.io.File;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Collection;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.DataStorageConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommand;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.G;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.lang.IgnitePredicate;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Test;
+import org.junit.runners.Parameterized.Parameter;
+import org.junit.runners.Parameterized.Parameters;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.cluster.ClusterState.ACTIVE;
+import static org.apache.ignite.internal.commandline.CommandHandler.EXIT_CODE_OK;
+import static org.apache.ignite.internal.processors.cache.persistence.snapshot.AbstractSnapshotSelfTest.snp;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Test for the command '--snapshot list'. */
+public class GridCommandHandlerListSnapshotTest extends GridCommandHandlerAbstractTest {
+    /** External storage path. */
+    private static final String EXT_STORAGE_PATH = "extStorage";
+
+    /** Flag to use {@link DataStorageConfiguration#setExtraSnapshotPaths(String...)}. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@code extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Consistent id postfix. */
+    private @Nullable String cstIdPostfix = "";
+
+    /** Flag setting the usage of a custom snapshot path. */
+    @Parameter(1)
+    public boolean customPath;
+
+    /** Flag setting the usage of dedicated, own node working directories. */
+    @Parameter(2)
+    public boolean separatedWorkDir;
+
+    /** Flag to add external server node after the snapshot creation. */
+    @Parameter(3)
+    public boolean addExtraSrvr;
+
+    /** Number of incremental snapshots to add to the main test snapshots. */
+    @Parameter(4)
+    public int incCnt;
+
+    /** */
+    @Parameters(name = "cmdHnd={0}, customPath={1}, ownWorkDir={2}, addExtraSrvr={3}, incCnt={4}")
+    public static Collection<?> parameters() {
+        return GridTestUtils.cartesianProduct(
+            commandHandlers(),
+            F.asList(false, true), // Custom snapshot path.
+            F.asList(false, true), // Separated (own) work directories.
+            F.asList(false, true), // Add a server node after the snapshot creation.
+            F.asList(0, 2) // Number of incremental snapshots.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        super.afterTest();
+
+        stopAllGrids();
+
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTest() throws Exception {
+        super.beforeTest();
+
+        /** Handy if test running is interrupted and {@link #afterTest()} isn't invoked. */
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
+
+        String workDir = separatedWorkDir
+            ? new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath()
+            : U.defaultWorkDirectory();
+
+        cfg.setWorkDirectory(workDir);
+
+        cfg.getDataStorageConfiguration().setWalCompactionEnabled(incCnt > 0);
+
+        if (extStorages) {
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                workDir + File.separator,
+                workDir + File.separator + EXT_STORAGE_PATH
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", EXT_STORAGE_PATH);
+        }
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override public String getTestIgniteInstanceName() {
+        return super.getTestIgniteInstanceName() + cstIdPostfix;
+    }
+
+    /** */
+    @Test
+    public void testNoSnapshots() throws Exception {
+        doTestSnapshotsLists(0, false, false);
+    }
+
+    /** */
+    @Test
+    public void testSingleSnapshot() throws Exception {
+        doTestSnapshotsLists(1, false, false);
+    }
+
+    /** */
+    @Test
+    public void testSeveralSnapshots() throws Exception {
+        doTestSnapshotsLists(3, false, false);
+    }
+
+    /** */
+    @Test
+    public void testOneNodeMisses() throws Exception {
+        // Doesn't matter here.
+//        assumeFalse(incCnt > 0);
+        // Keeps just one node not seeing the snapshot.
+        assumeTrue(separatedWorkDir);
+
+        doTestSnapshotsLists(2, true, false);
+    }
+
+    /** */
+    @Test
+    public void testExternalStorages() throws Exception {
+        // External storages are required to be the same as configured in the node's PDS storages. Thus, we skip different work folders.
+        // Also, extra snapshot storages aren't used if snapshot is created with a custom path.
+        assumeFalse(separatedWorkDir || customPath);
+
+        extStorages = true;
+
+        doTestSnapshotsLists(2, false, false);
+    }
+
+    /** */
+    @Test
+    public void testChangedConsistentId() throws Exception {
+        // In new created working directories there will be obviously no snapshots.
+        assumeFalse(separatedWorkDir);
+
+        doTestSnapshotsLists(1, false, true);
+    }
+
+    /** */
+    private void doTestSnapshotsLists(int snpCnt, boolean deleteOnOneNode, boolean restartWithChangedCstIds) throws Exception {

Review Comment:
   You write very hard for human reading tests ! Look you have 5 parametrized class and also input test params and common test part without detailed descriprion, if i can parse it for now - seems need huge time to parse it in future if smth. goes wrong, I suggest to cut this common test into parts of self documented ones, or if you hardly against my suggestion - add extended description.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6413278487

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotCheckCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotDeleteCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotRestoreCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324438893


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+            // Read local snapshots.
+            List<T2<SnapshotFileTree, Long>> locSnps = findLocalSnapshots(arg.src());
+
+            if (locSnps.isEmpty())
+                return new SnapshotListJobResult(Collections.emptyMap());
+
+            // Main sizes of the local snapshots (per snapshot name).
+            Map<String, Long> sizes = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional incremental parts descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> incs = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional external storages descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> extStors = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+
+            // Snapshot sizes, incremental parts and external storages futures.
+            List<Future<?>> futs = new ArrayList<>(locSnps.size() * 3);
+            // Names of snapshots with read failures.
+            Set<String> failedSnps = ConcurrentHashMap.newKeySet(locSnps.size() / 2);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            // Descriptions of the local snapshots: main sizes, external storages and incremental parts.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                SnapshotFileTree sft = snpPair.get1();
+                String snpName = sft.name();
+
+                // Future for the snapshot main size. It is the largest part of the job, thus it goes first.
+                Future<?> fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        sizes.put(snpName, calculateDirectorySize(sft.root()));
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to calculate snapshot's size, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional incremental parts.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo incDesc = incrementals(sft);
+
+                        if (incDesc != null)
+                            incs.put(snpName, incDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's incremental parts, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional external storages.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo extDesc = externalStorages(sft);
+
+                        if (extDesc != null)
+                            extStors.put(snpName, extDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's external storages, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+            }
+
+            // Wait for the futures.
+            for (Future<?> fut : futs) {
+                try {
+                    fut.get();
+                }
+                catch (ExecutionException e) {
+                    throw new IgniteException("Failed to read local node's snapshots.", e);
+                }
+                catch (InterruptedException e) {
+                    Thread.currentThread().interrupt();
+
+                    throw new IgniteException("Interrupted while reading local snapshots.", e);
+                }
+            }
+
+            // Result snapshot descriptions.
+            Map<String, SnapshotListJobResult.SnapshotInfo> resMap = new ConcurrentHashMap<>(locSnps.size() - failedSnps.size(), 1.0f);
+
+            // Reduce results.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                String snpName = snpPair.get1().name();
+
+                if (failedSnps.contains(snpName))
+                    continue;
+
+                Long size = sizes.get(snpName);
+
+                // The size must be calculated unless the snapshot has failed.
+                assert size != null;
+
+                SnapshotListJobResult.SnapshotInfo snpDesc = new SnapshotListJobResult.SnapshotInfo(
+                    size,
+                    snpPair.get2(),
+                    extStors.get(snpName),
+                    incs.get(snpName)
+                );
+
+                resMap.put(snpName, snpDesc);
+            }
+
+            return new SnapshotListJobResult(resMap);
+        }
+
+        /** */
+        private @Nullable SnapshotListJobResult.SnapshotInfo externalStorages(SnapshotFileTree sft) {
+            int extStoragesCnt = 0;
+            long extStoragesSize = 0;
+
+            for (File extraStorage : sft.allStorages().toList()) {
+                if (sft.nodeStorage().equals(extraStorage))
+                    continue;
+
+                try {
+                    extStoragesSize += calculateDirectorySize(extraStorage);
+                }
+                catch (Exception e) {
+                    log.warning("Failed to calculate snapshot's external storage size, storage ignored [extraStorage='" +
+                        extraStorage + "']", e);
+
+                    continue;
+                }
+
+                extStoragesCnt++;
+            }
+
+            return extStoragesCnt == 0 ? null : new SnapshotListJobResult.SnapshotInfo(extStoragesCnt, extStoragesSize);
+        }
+
+        /** @return Snapshot file tree and creation time from the snapshot metadata. */
+        private List<T2<SnapshotFileTree, Long>> findLocalSnapshots(@Nullable String snpPath) {
+            // The tree is used only to extract the snapshots root directory. Such snapshot name is actually unallowed.
+            File[] dirsToParse = new SnapshotFileTree(ignite.context(), "snp", snpPath).root().getParentFile().listFiles();
+
+            if (F.isEmpty(dirsToParse))
+                return Collections.emptyList();
+
+            List<Future<T2<SnapshotFileTree, Long>>> futs = new ArrayList<>(dirsToParse.length);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();

Review Comment:
   Because there can be many snapshots, and they can be large. For snapshots, we have a dedicated thread pool — why not use it? And the operation is not cancellable. Cache/txs do not use the snapshot thread pool, of course.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324552065


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/CoreFeatureRegistry.java:
##########
@@ -93,4 +93,7 @@
 public class CoreFeatureRegistry {
     /** */
     public static final IgniteFeature ROLLING_UPGRADE_FEATURE = new IgniteCoreFeature(0);
+
+    /** */
+    public static final IgniteFeature SNAPSHOT_LIST_FEATURE = new IgniteCoreFeature(1);

Review Comment:
   Will change when merging the master branch, of course. That would be #3, I guess. There will be no problem. The number will not interfere.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4323554370


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324209819


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {

Review Comment:
   Done. Delete here, is not necessary.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6301246828

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsSingleMessage.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "zstan (via GitHub)" <gi...@apache.org>.
zstan commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4345606848


##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListTest.java:
##########
@@ -0,0 +1,936 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.io.Serializable;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Stream;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.lang.IgniteFuture;
+import org.apache.ignite.plugin.AbstractTestPluginProvider;
+import org.apache.ignite.plugin.PluginConfiguration;
+import org.apache.ignite.plugin.PluginContext;
+import org.apache.ignite.plugin.PluginProvider;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runners.Parameterized;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.configuration.IgniteConfiguration.DFLT_SNAPSHOT_THREAD_POOL_SIZE;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.cartesianProduct;
+import static org.apache.ignite.testframework.GridTestUtils.runAsync;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Cluster-wide snapshot list procedure tests. */
+public class IgniteClusterSnapshotListTest extends AbstractSnapshotSelfTest {
+    /** Number of cache keys to pre-create at node startup. */
+    private static final int CACHE_KEYS_RANGE = 15;
+
+    /** Number of partitions of the test cache. */
+    private static final int CACHE_PARTITIONS_COUNT = 4;
+
+    /** Flag showing that the filesystem supports POSIX permissions. */
+    private static boolean posixPermissions;
+
+    /** Size of the snapshot utility thread pool. */
+    @Parameterized.Parameter(2)
+    public int snpThrdPoolSz;
+
+    /** */
+    private PluginProvider<PluginConfiguration> pluginProvider;
+
+    /** Flag to spread the test cache data over the external storages. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@link #extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Parameters. */
+    @Parameterized.Parameters(name = "encryption={0}, onlyPrimary={1}, snpThrdPoolSz={2}")
+    public static Collection<Object[]> params() {
+        return cartesianProduct(
+            encryptionParameters(), // Encryption.
+            F.asList(false, true), // Only primary.
+            F.asList(DFLT_SNAPSHOT_THREAD_POOL_SIZE, 1) // Snapshots thread pool size.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName)
+            .setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        if (pluginProvider != null)
+            cfg.setPluginProviders(pluginProvider);
+
+        if (extStorages) {
+            // External storage paths must be identical on all the nodes (a cache has the single storage paths
+            // setting). Thus, a shared work directory is used, like in GridCommandHandlerListSnapshotTest.
+            cfg.setWorkDirectory(U.defaultWorkDirectory());
+
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                U.defaultWorkDirectory() + File.separator,
+                U.defaultWorkDirectory() + File.separator + "extStorage"
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", "extStorage");
+        }
+
+        cfg.setSnapshotThreadPoolSize(snpThrdPoolSz);
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        super.cleanPersistenceDir();
+
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTestsStarted() throws Exception {
+        super.beforeTestsStarted();
+
+        File workDir = new File(U.defaultWorkDirectory());
+
+        assertTrue(workDir.exists());
+
+        try {
+            Files.getPosixFilePermissions(workDir.toPath());
+
+            posixPermissions = true;
+        }
+        catch (UnsupportedOperationException ignored) {
+            // No-op.
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected <K, V> CacheConfiguration<K, V> txCacheConfig(CacheConfiguration<K, V> ccfg) {
+        // Speeds up the tests.
+        ccfg = super.txCacheConfig(ccfg)
+            .setAffinity(new RendezvousAffinityFunction(false, CACHE_PARTITIONS_COUNT))
+            .setBackups(1);
+
+        if (extStorages) {
+            assert !F.isEmpty(extStoragePaths);
+
+            ccfg.setStoragePaths(extStoragePaths);
+        }
+
+        return ccfg;
+    }
+
+    /** */
+    @Test
+    public void testConcurrentCreation() throws Exception {
+        // The test uses the thread blocking and conditional waiting. Won't proceed with 1 thread.
+        assumeTrue(snpThrdPoolSz > 1);
+
+        int grids = 3;
+        int testNodeIdx = 1;
+        int testNodeOrder = testNodeIdx + 1;
+
+        CountDownLatch beginSnpCreation = new CountDownLatch(grids);
+        CountDownLatch proceedSnpCreation = new CountDownLatch(1);
+
+        // Delays snapshot creation after its metadata is written.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public <M extends Serializable> void storeSnapshotMeta(M meta, File smf) {
+                            super.storeSnapshotMeta(meta, smf);
+
+                            beginSnpCreation.countDown();
+
+                            if (((IgniteEx)ctx.grid()).localNode().order() == testNodeOrder) {
+                                try {
+                                    assertTrue(proceedSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        IgniteFuture<Void> createSnpFut = snp(grid(0)).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary);
+
+        assertTrue(beginSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        SnapshotListTaskResult lstOpRes = listSnapshots(grid(0));
+
+        int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+        assertEquals(grids, snpsCnt);
+
+        SnapshotFileTree testSft = new SnapshotFileTree(grid(testNodeIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(SnapshotListTask.calculateDirectorySize(testSft.root()) > 0L);
+
+        proceedSnpCreation.countDown();
+
+        createSnpFut.get(getTestTimeout());
+    }
+
+    /** */
+    @Test
+    public void testCompletelyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(true);
+    }
+
+    /** */
+    @Test
+    public void testPartlyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(false);
+    }
+
+    /** */
+    private void doTestDeletionAfterMetaRead(boolean completeDeletion) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        CountDownLatch metaReadBeginLatch = new CountDownLatch(grids);
+        CountDownLatch metaReadProceedLatch = new CountDownLatch(1);
+
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft, boolean failIfCantRead) {
+                            metaReadBeginLatch.countDown();
+
+                            if (cctx.localNode().order() == testGridIdx + 1) {
+                                try {
+                                    assertTrue(metaReadProceedLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        long testSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+        assertTrue(testSnpSz > 0);
+
+        IgniteInternalFuture<SnapshotListTaskResult> lstOpFut = runAsync(() -> listSnapshots(grid(0)));
+
+        assertTrue(metaReadBeginLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        if (completeDeletion)
+            assertTrue(testSnpFt.root().exists() && U.delete(testSnpFt.root()) && !testSnpFt.root().exists());
+        else
+            assertTrue(testSnpFt.nodeStorage().exists() && U.delete(testSnpFt.nodeStorage()) && !testSnpFt.nodeStorage().exists());
+
+        metaReadProceedLatch.countDown();
+
+        SnapshotListTaskResult lstOpRes = lstOpFut.get();
+
+        if (completeDeletion) {
+            int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+            assertEquals(grids - 1, snpsCnt);
+        }
+        else {
+            int snpsCnt = 0;
+            boolean victimNodeFound = false;
+
+            for (int i = 0; i < lstOpRes.nodesIds().length; i++) {
+                UUID nid = lstOpRes.nodesIds()[i];
+
+                Map<String, SnapshotListJobResult.SnapshotInfo> nodeSnps = lstOpRes.nodesSnapshots()[i].snapshots();
+
+                snpsCnt += nodeSnps.size();
+
+                if (nid.equals(grid(testGridIdx).localNode().id())) {
+                    victimNodeFound = true;
+
+                    assertEquals(1, nodeSnps.size());
+
+                    long curSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+                    assertTrue(curSnpSz < testSnpSz);
+                }
+            }
+
+            assertTrue(victimNodeFound);
+            assertEquals(grids, snpsCnt);
+        }
+    }
+
+    /** */
+    @Test
+    public void testMissingSnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(true);
+    }
+
+    /** */
+    @Test
+    public void testEmptySnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(false);
+    }
+
+    /** */
+    private void doTestWrongSnapshotPath(boolean missing) throws Exception {
+        // Doesn't matter here, speeds up the tests.
+        assumeFalse(snpThrdPoolSz < 2);

Review Comment:
   First - you remove "assumeFalse(snpThrdPoolSz < 2);" and after you write this comment ) My - become unrelevant after your changes )



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "zstan (via GitHub)" <gi...@apache.org>.
zstan commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4345739553


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");
+
+            printer.accept("\tNode '%s' [uuid=%s]:".formatted(res.consistentIds()[nodeIdx], res.nodesIds()[nodeIdx]));
+
+            SnapshotListJobResult nodeResult = res.nodesSnapshots()[nodeIdx];
+
+            if (nodeResult.snapshots().isEmpty()) {
+                printer.accept("\t\t" + NO_SNAPSHOTS);
+
+                continue;
+            }
+
+            // Sorted snapshots names for better user reading.
+            new TreeMap<>(nodeResult.snapshots()).forEach((snpName, snpInfo) -> {
+                printer.accept("\t\tSnapshot '%s': totalSize=%s (%db), created='%s' (epoch=%d)".formatted(

Review Comment:
   from discussion: need to investigate common usage timporal output format, probably : dd.MM.YYYY - HH:MM



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "zstan (via GitHub)" <gi...@apache.org>.
zstan commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4347005579


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");

Review Comment:
   Seems we agreed with table with columns view here 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4323588674


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");

Review Comment:
   The comment removed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4323691144


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");
+
+            printer.accept("\tNode '%s' [uuid=%s]:".formatted(res.consistentIds()[nodeIdx], res.nodesIds()[nodeIdx]));
+
+            SnapshotListJobResult nodeResult = res.nodesSnapshots()[nodeIdx];
+
+            if (nodeResult.snapshots().isEmpty()) {
+                printer.accept("\t\t" + NO_SNAPSHOTS);
+
+                continue;
+            }
+
+            // Sorted snapshots names for better user reading.
+            new TreeMap<>(nodeResult.snapshots()).forEach((snpName, snpInfo) -> {
+                printer.accept("\t\tSnapshot '%s': totalSize=%s (%db), created='%s' (epoch=%d)".formatted(

Review Comment:
   Good question. Here's the thinking:
   
   **Exact date**: tying the output to a hardcoded format isn't a good practice actually (it varies by region). Adding a config option would bloat this ticket. If anyone want othe date format, epoch time
   is always available. I was actually thinking of a `verbose` flag that would return epoch time and the byte size.
   
   **Exact size:** familiar commands across OSes give you both a friendly and an exact view. I kept it, but I'm fine dropping the byte value if the team prefers.
   
   WDYT?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324209819


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {

Review Comment:
   Done. Moved to `GridDiscoveryManager` (`findConsistentId()`)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4323453281


##########
docs/_docs/snapshots/snapshots.adoc:
##########
@@ -287,9 +287,49 @@ control.(sh|bat) --snapshot restore snapshot_09062021 --groups cache-group1,cach
 control.(sh|bat) --snapshot restore snapshot_09062021 --increment 1
 ----
 
+== Listing Snapshots
+
+To list snapshots created on the cluster, use the following command.
+
+[tabs]
+--
+tab:Unix[]
+[source,shell]
+----
+# List all snapshots in the default snapshot directory.
+control.sh --snapshot list
+
+# List all snapshots in the "/tmp/ignite/snapshots" folder.
+control.sh --snapshot list --src /tmp/ignite/snapshots
+----
+
+tab:Windows[]
+[source,shell]
+----
+# List all snapshots in the default snapshot directory.
+control.bat --snapshot list
+
+# List all snapshots in the "C:\tmp\ignite\snapshots" folder.
+control.bat --snapshot list --src C:\tmp\ignite\snapshots
+----
+--
+
+=== List operation notes
+
+* The list operation is read-only. It does not modify any snapshot data.
+* A snapshot is detected by its metadata. If the metadata is missing or cannot be read, the snapshot is ignored.
+* The operation reads snapshot metadata and incremental snapshot metadata. On most platforms, this does not interfere
+with concurrent modification or deletion of snapshots. On Windows, deleting a snapshot at the exact moment its
+metadata is being read might be incomplete and require repeating or manual result control. The probability of this is low.
+* Listing and size calculation do not validate snapshots or verify checksums.
+* If a snapshot is being created or deleted, or is corrupted, it might be skipped or reported partially
+(for example, with an underestimated size). The operation reads what it can and ignores most errors.
+* The list operation is expected to be fast, so, unlike snapshot creation or restoration, it cannot be canceled and has no status.
+
+
 == Getting Snapshot Operation Status
 
-The status of the current snapshot operation in the cluster can be obtained using the `control.sh|bat` script or JMX interface:
+The status of the create/restore snapshot operations in the cluster can be obtained using the `control.sh|bat` script or JMX interface:

Review Comment:
   > I`d prefer a previous version of this line, why we need to change it ?
   
   This emphasizes that the status is available only for creation and restoration — not for any operation, not for the lists (and not for deletion either).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324438893


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+            // Read local snapshots.
+            List<T2<SnapshotFileTree, Long>> locSnps = findLocalSnapshots(arg.src());
+
+            if (locSnps.isEmpty())
+                return new SnapshotListJobResult(Collections.emptyMap());
+
+            // Main sizes of the local snapshots (per snapshot name).
+            Map<String, Long> sizes = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional incremental parts descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> incs = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional external storages descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> extStors = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+
+            // Snapshot sizes, incremental parts and external storages futures.
+            List<Future<?>> futs = new ArrayList<>(locSnps.size() * 3);
+            // Names of snapshots with read failures.
+            Set<String> failedSnps = ConcurrentHashMap.newKeySet(locSnps.size() / 2);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            // Descriptions of the local snapshots: main sizes, external storages and incremental parts.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                SnapshotFileTree sft = snpPair.get1();
+                String snpName = sft.name();
+
+                // Future for the snapshot main size. It is the largest part of the job, thus it goes first.
+                Future<?> fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        sizes.put(snpName, calculateDirectorySize(sft.root()));
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to calculate snapshot's size, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional incremental parts.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo incDesc = incrementals(sft);
+
+                        if (incDesc != null)
+                            incs.put(snpName, incDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's incremental parts, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional external storages.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo extDesc = externalStorages(sft);
+
+                        if (extDesc != null)
+                            extStors.put(snpName, extDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's external storages, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+            }
+
+            // Wait for the futures.
+            for (Future<?> fut : futs) {
+                try {
+                    fut.get();
+                }
+                catch (ExecutionException e) {
+                    throw new IgniteException("Failed to read local node's snapshots.", e);
+                }
+                catch (InterruptedException e) {
+                    Thread.currentThread().interrupt();
+
+                    throw new IgniteException("Interrupted while reading local snapshots.", e);
+                }
+            }
+
+            // Result snapshot descriptions.
+            Map<String, SnapshotListJobResult.SnapshotInfo> resMap = new ConcurrentHashMap<>(locSnps.size() - failedSnps.size(), 1.0f);
+
+            // Reduce results.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                String snpName = snpPair.get1().name();
+
+                if (failedSnps.contains(snpName))
+                    continue;
+
+                Long size = sizes.get(snpName);
+
+                // The size must be calculated unless the snapshot has failed.
+                assert size != null;
+
+                SnapshotListJobResult.SnapshotInfo snpDesc = new SnapshotListJobResult.SnapshotInfo(
+                    size,
+                    snpPair.get2(),
+                    extStors.get(snpName),
+                    incs.get(snpName)
+                );
+
+                resMap.put(snpName, snpDesc);
+            }
+
+            return new SnapshotListJobResult(resMap);
+        }
+
+        /** */
+        private @Nullable SnapshotListJobResult.SnapshotInfo externalStorages(SnapshotFileTree sft) {
+            int extStoragesCnt = 0;
+            long extStoragesSize = 0;
+
+            for (File extraStorage : sft.allStorages().toList()) {
+                if (sft.nodeStorage().equals(extraStorage))
+                    continue;
+
+                try {
+                    extStoragesSize += calculateDirectorySize(extraStorage);
+                }
+                catch (Exception e) {
+                    log.warning("Failed to calculate snapshot's external storage size, storage ignored [extraStorage='" +
+                        extraStorage + "']", e);
+
+                    continue;
+                }
+
+                extStoragesCnt++;
+            }
+
+            return extStoragesCnt == 0 ? null : new SnapshotListJobResult.SnapshotInfo(extStoragesCnt, extStoragesSize);
+        }
+
+        /** @return Snapshot file tree and creation time from the snapshot metadata. */
+        private List<T2<SnapshotFileTree, Long>> findLocalSnapshots(@Nullable String snpPath) {
+            // The tree is used only to extract the snapshots root directory. Such snapshot name is actually unallowed.
+            File[] dirsToParse = new SnapshotFileTree(ignite.context(), "snp", snpPath).root().getParentFile().listFiles();
+
+            if (F.isEmpty(dirsToParse))
+                return Collections.emptyList();
+
+            List<Future<T2<SnapshotFileTree, Long>>> futs = new ArrayList<>(dirsToParse.length);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();

Review Comment:
   Because there can be many snapshots, and they can be large. For snapshots, we have a dedicated thread pool — why not use it? And the operation is not cancellable. Cache/txs do not use the snapshot thread pool, of course. Also, the snapshot pool size is configurable.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4375618875


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+            // Read local snapshots.
+            List<T2<SnapshotFileTree, Long>> locSnps = findLocalSnapshots(arg.src());
+
+            if (locSnps.isEmpty())
+                return new SnapshotListJobResult(Collections.emptyMap());
+
+            // Main sizes of the local snapshots (per snapshot name).
+            Map<String, Long> sizes = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional incremental parts descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> incs = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional external storages descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> extStors = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+
+            // Snapshot sizes, incremental parts and external storages futures.
+            List<Future<?>> futs = new ArrayList<>(locSnps.size() * 3);
+            // Names of snapshots with read failures.
+            Set<String> failedSnps = ConcurrentHashMap.newKeySet(locSnps.size() / 2);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            // Descriptions of the local snapshots: main sizes, external storages and incremental parts.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                SnapshotFileTree sft = snpPair.get1();
+                String snpName = sft.name();
+
+                // Future for the snapshot main size. It is the largest part of the job, thus it goes first.
+                Future<?> fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        sizes.put(snpName, calculateDirectorySize(sft.root()));
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to calculate snapshot's size, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional incremental parts.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo incDesc = incrementals(sft);
+
+                        if (incDesc != null)
+                            incs.put(snpName, incDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's incremental parts, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional external storages.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo extDesc = externalStorages(sft);
+
+                        if (extDesc != null)
+                            extStors.put(snpName, extDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's external storages, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+            }
+
+            // Wait for the futures.
+            for (Future<?> fut : futs) {
+                try {
+                    fut.get();
+                }
+                catch (ExecutionException e) {
+                    throw new IgniteException("Failed to read local node's snapshots.", e);
+                }
+                catch (InterruptedException e) {
+                    Thread.currentThread().interrupt();
+
+                    throw new IgniteException("Interrupted while reading local snapshots.", e);
+                }
+            }
+
+            // Result snapshot descriptions.
+            Map<String, SnapshotListJobResult.SnapshotInfo> resMap = new ConcurrentHashMap<>(locSnps.size() - failedSnps.size(), 1.0f);
+
+            // Reduce results.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                String snpName = snpPair.get1().name();
+
+                if (failedSnps.contains(snpName))
+                    continue;
+
+                Long size = sizes.get(snpName);
+
+                // The size must be calculated unless the snapshot has failed.
+                assert size != null;
+
+                SnapshotListJobResult.SnapshotInfo snpDesc = new SnapshotListJobResult.SnapshotInfo(
+                    size,
+                    snpPair.get2(),
+                    extStors.get(snpName),
+                    incs.get(snpName)
+                );
+
+                resMap.put(snpName, snpDesc);
+            }
+
+            return new SnapshotListJobResult(resMap);
+        }
+
+        /** */
+        private @Nullable SnapshotListJobResult.SnapshotInfo externalStorages(SnapshotFileTree sft) {
+            int extStoragesCnt = 0;
+            long extStoragesSize = 0;
+
+            for (File extraStorage : sft.allStorages().toList()) {
+                if (sft.nodeStorage().equals(extraStorage))
+                    continue;
+
+                try {
+                    extStoragesSize += calculateDirectorySize(extraStorage);
+                }
+                catch (Exception e) {
+                    log.warning("Failed to calculate snapshot's external storage size, storage ignored [extraStorage='" +
+                        extraStorage + "']", e);
+
+                    continue;
+                }
+
+                extStoragesCnt++;
+            }
+
+            return extStoragesCnt == 0 ? null : new SnapshotListJobResult.SnapshotInfo(extStoragesCnt, extStoragesSize);
+        }
+
+        /** @return Snapshot file tree and creation time from the snapshot metadata. */
+        private List<T2<SnapshotFileTree, Long>> findLocalSnapshots(@Nullable String snpPath) {
+            // The tree is used only to extract the snapshots root directory. Such snapshot name is actually unallowed.
+            File[] dirsToParse = new SnapshotFileTree(ignite.context(), "snp", snpPath).root().getParentFile().listFiles();

Review Comment:
   I would need to create a `_SnapshotFileTree_`, pass it to `root()`, and similarly pass a snapshot name either.
   
   I don’t see any pros. Here we use exactly `SnapshotFileTree`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4327255111


##########
modules/control-utility/src/test/java/org/apache/ignite/util/GridCommandHandlerListSnapshotTest.java:
##########
@@ -0,0 +1,348 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.util;
+
+import java.io.File;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Collection;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.DataStorageConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommand;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.G;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.lang.IgnitePredicate;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Test;
+import org.junit.runners.Parameterized.Parameter;
+import org.junit.runners.Parameterized.Parameters;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.cluster.ClusterState.ACTIVE;
+import static org.apache.ignite.internal.commandline.CommandHandler.EXIT_CODE_OK;
+import static org.apache.ignite.internal.processors.cache.persistence.snapshot.AbstractSnapshotSelfTest.snp;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Test for the command '--snapshot list'. */
+public class GridCommandHandlerListSnapshotTest extends GridCommandHandlerAbstractTest {
+    /** External storage path. */
+    private static final String EXT_STORAGE_PATH = "extStorage";
+
+    /** Flag to use {@link DataStorageConfiguration#setExtraSnapshotPaths(String...)}. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@code extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Consistent id postfix. */
+    private @Nullable String cstIdPostfix = "";
+
+    /** Flag setting the usage of a custom snapshot path. */
+    @Parameter(1)
+    public boolean customPath;
+
+    /** Flag setting the usage of dedicated, own node working directories. */
+    @Parameter(2)
+    public boolean separatedWorkDir;
+
+    /** Flag to add external server node after the snapshot creation. */
+    @Parameter(3)
+    public boolean addExtraSrvr;
+
+    /** Number of incremental snapshots to add to the main test snapshots. */
+    @Parameter(4)
+    public int incCnt;
+
+    /** */
+    @Parameters(name = "cmdHnd={0}, customPath={1}, ownWorkDir={2}, addExtraSrvr={3}, incCnt={4}")
+    public static Collection<?> parameters() {
+        return GridTestUtils.cartesianProduct(
+            commandHandlers(),
+            F.asList(false, true), // Custom snapshot path.
+            F.asList(false, true), // Separated (own) work directories.
+            F.asList(false, true), // Add a server node after the snapshot creation.
+            F.asList(0, 2) // Number of incremental snapshots.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        super.afterTest();
+
+        stopAllGrids();
+
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTest() throws Exception {
+        super.beforeTest();
+
+        /** Handy if test running is interrupted and {@link #afterTest()} isn't invoked. */
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
+
+        String workDir = separatedWorkDir
+            ? new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath()
+            : U.defaultWorkDirectory();
+
+        cfg.setWorkDirectory(workDir);
+
+        cfg.getDataStorageConfiguration().setWalCompactionEnabled(incCnt > 0);
+
+        if (extStorages) {
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                workDir + File.separator,
+                workDir + File.separator + EXT_STORAGE_PATH
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", EXT_STORAGE_PATH);
+        }
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override public String getTestIgniteInstanceName() {
+        return super.getTestIgniteInstanceName() + cstIdPostfix;
+    }
+
+    /** */
+    @Test
+    public void testNoSnapshots() throws Exception {
+        doTestSnapshotsLists(0, false, false);
+    }
+
+    /** */
+    @Test
+    public void testSingleSnapshot() throws Exception {
+        doTestSnapshotsLists(1, false, false);
+    }
+
+    /** */
+    @Test
+    public void testSeveralSnapshots() throws Exception {
+        doTestSnapshotsLists(3, false, false);
+    }
+
+    /** */
+    @Test
+    public void testOneNodeMisses() throws Exception {
+        // Doesn't matter here.
+//        assumeFalse(incCnt > 0);
+        // Keeps just one node not seeing the snapshot.
+        assumeTrue(separatedWorkDir);
+
+        doTestSnapshotsLists(2, true, false);
+    }
+
+    /** */
+    @Test
+    public void testExternalStorages() throws Exception {
+        // External storages are required to be the same as configured in the node's PDS storages. Thus, we skip different work folders.
+        // Also, extra snapshot storages aren't used if snapshot is created with a custom path.
+        assumeFalse(separatedWorkDir || customPath);
+
+        extStorages = true;
+
+        doTestSnapshotsLists(2, false, false);
+    }
+
+    /** */
+    @Test
+    public void testChangedConsistentId() throws Exception {
+        // In new created working directories there will be obviously no snapshots.
+        assumeFalse(separatedWorkDir);
+
+        doTestSnapshotsLists(1, false, true);
+    }
+
+    /** */
+    private void doTestSnapshotsLists(int snpCnt, boolean deleteOnOneNode, boolean restartWithChangedCstIds) throws Exception {

Review Comment:
   Done. Applied a simplification, params reduction and added a bunch of comments, test scenarion description.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6372397380

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4376997400


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',

Review Comment:
   Done. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "zstan (via GitHub)" <gi...@apache.org>.
zstan commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4310768558


##########
docs/_docs/snapshots/snapshots.adoc:
##########
@@ -287,9 +287,49 @@ control.(sh|bat) --snapshot restore snapshot_09062021 --groups cache-group1,cach
 control.(sh|bat) --snapshot restore snapshot_09062021 --increment 1
 ----
 
+== Listing Snapshots
+
+To list snapshots created on the cluster, use the following command.
+
+[tabs]
+--
+tab:Unix[]
+[source,shell]
+----
+# List all snapshots in the default snapshot directory.
+control.sh --snapshot list
+
+# List all snapshots in the "/tmp/ignite/snapshots" folder.
+control.sh --snapshot list --src /tmp/ignite/snapshots
+----
+
+tab:Windows[]
+[source,shell]
+----
+# List all snapshots in the default snapshot directory.
+control.bat --snapshot list
+
+# List all snapshots in the "C:\tmp\ignite\snapshots" folder.
+control.bat --snapshot list --src C:\tmp\ignite\snapshots
+----
+--
+
+=== List operation notes
+
+* The list operation is read-only. It does not modify any snapshot data.
+* A snapshot is detected by its metadata. If the metadata is missing or cannot be read, the snapshot is ignored.
+* The operation reads snapshot metadata and incremental snapshot metadata. On most platforms, this does not interfere
+with concurrent modification or deletion of snapshots. On Windows, deleting a snapshot at the exact moment its
+metadata is being read might be incomplete and require repeating or manual result control. The probability of this is low.
+* Listing and size calculation do not validate snapshots or verify checksums.
+* If a snapshot is being created or deleted, or is corrupted, it might be skipped or reported partially
+(for example, with an underestimated size). The operation reads what it can and ignores most errors.
+* The list operation is expected to be fast, so, unlike snapshot creation or restoration, it cannot be canceled and has no status.
+
+
 == Getting Snapshot Operation Status
 
-The status of the current snapshot operation in the cluster can be obtained using the `control.sh|bat` script or JMX interface:
+The status of the create/restore snapshot operations in the cluster can be obtained using the `control.sh|bat` script or JMX interface:

Review Comment:
   I\`d prefer a previous version of this line, why we need to change it ?



##########
modules/control-utility/src/test/java/org/apache/ignite/util/GridCommandHandlerListSnapshotTest.java:
##########
@@ -0,0 +1,348 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.util;
+
+import java.io.File;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Collection;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.DataStorageConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommand;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.G;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.lang.IgnitePredicate;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Test;
+import org.junit.runners.Parameterized.Parameter;
+import org.junit.runners.Parameterized.Parameters;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.cluster.ClusterState.ACTIVE;
+import static org.apache.ignite.internal.commandline.CommandHandler.EXIT_CODE_OK;
+import static org.apache.ignite.internal.processors.cache.persistence.snapshot.AbstractSnapshotSelfTest.snp;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Test for the command '--snapshot list'. */
+public class GridCommandHandlerListSnapshotTest extends GridCommandHandlerAbstractTest {
+    /** External storage path. */
+    private static final String EXT_STORAGE_PATH = "extStorage";
+
+    /** Flag to use {@link DataStorageConfiguration#setExtraSnapshotPaths(String...)}. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@code extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Consistent id postfix. */
+    private @Nullable String cstIdPostfix = "";
+
+    /** Flag setting the usage of a custom snapshot path. */
+    @Parameter(1)
+    public boolean customPath;
+
+    /** Flag setting the usage of dedicated, own node working directories. */
+    @Parameter(2)
+    public boolean separatedWorkDir;
+
+    /** Flag to add external server node after the snapshot creation. */
+    @Parameter(3)
+    public boolean addExtraSrvr;
+
+    /** Number of incremental snapshots to add to the main test snapshots. */
+    @Parameter(4)
+    public int incCnt;
+
+    /** */
+    @Parameters(name = "cmdHnd={0}, customPath={1}, ownWorkDir={2}, addExtraSrvr={3}, incCnt={4}")
+    public static Collection<?> parameters() {
+        return GridTestUtils.cartesianProduct(
+            commandHandlers(),
+            F.asList(false, true), // Custom snapshot path.
+            F.asList(false, true), // Separated (own) work directories.
+            F.asList(false, true), // Add a server node after the snapshot creation.
+            F.asList(0, 2) // Number of incremental snapshots.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        super.afterTest();
+
+        stopAllGrids();
+
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTest() throws Exception {
+        super.beforeTest();
+
+        /** Handy if test running is interrupted and {@link #afterTest()} isn't invoked. */
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
+
+        String workDir = separatedWorkDir
+            ? new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath()
+            : U.defaultWorkDirectory();
+
+        cfg.setWorkDirectory(workDir);
+
+        cfg.getDataStorageConfiguration().setWalCompactionEnabled(incCnt > 0);
+
+        if (extStorages) {
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                workDir + File.separator,
+                workDir + File.separator + EXT_STORAGE_PATH
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", EXT_STORAGE_PATH);
+        }
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override public String getTestIgniteInstanceName() {
+        return super.getTestIgniteInstanceName() + cstIdPostfix;
+    }
+
+    /** */
+    @Test
+    public void testNoSnapshots() throws Exception {
+        doTestSnapshotsLists(0, false, false);
+    }
+
+    /** */
+    @Test
+    public void testSingleSnapshot() throws Exception {
+        doTestSnapshotsLists(1, false, false);
+    }
+
+    /** */
+    @Test
+    public void testSeveralSnapshots() throws Exception {
+        doTestSnapshotsLists(3, false, false);
+    }
+
+    /** */
+    @Test
+    public void testOneNodeMisses() throws Exception {
+        // Doesn't matter here.
+//        assumeFalse(incCnt > 0);

Review Comment:
   redundant



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");
+
+            printer.accept("\tNode '%s' [uuid=%s]:".formatted(res.consistentIds()[nodeIdx], res.nodesIds()[nodeIdx]));
+
+            SnapshotListJobResult nodeResult = res.nodesSnapshots()[nodeIdx];
+
+            if (nodeResult.snapshots().isEmpty()) {
+                printer.accept("\t\t" + NO_SNAPSHOTS);
+
+                continue;
+            }
+
+            // Sorted snapshots names for better user reading.

Review Comment:
   ```suggestion
               // Sorted snapshot names for better readability.
   ```



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");
+
+            printer.accept("\tNode '%s' [uuid=%s]:".formatted(res.consistentIds()[nodeIdx], res.nodesIds()[nodeIdx]));
+
+            SnapshotListJobResult nodeResult = res.nodesSnapshots()[nodeIdx];
+
+            if (nodeResult.snapshots().isEmpty()) {
+                printer.accept("\t\t" + NO_SNAPSHOTS);
+
+                continue;
+            }
+
+            // Sorted snapshots names for better user reading.
+            new TreeMap<>(nodeResult.snapshots()).forEach((snpName, snpInfo) -> {
+                printer.accept("\t\tSnapshot '%s': totalSize=%s (%db), created='%s' (epoch=%d)".formatted(
+                    snpName,
+                    U.humanReadableByteCount(snpInfo.size()),
+                    snpInfo.size(),
+                    DATE_FORMATTER.format(Instant.ofEpochMilli(snpInfo.date())),
+                    snpInfo.date()
+                ));
+
+                SnapshotListJobResult.SnapshotInfo extStors = snpInfo.externalStorages();
+                SnapshotListJobResult.SnapshotInfo incs = snpInfo.incrementals();
+
+                if (extStors != null) {
+                    printer.accept("\t\t\texternal storages: cnt=%d, size=%s (%db)".formatted(
+                        extStors.number(),
+                        U.humanReadableByteCount(extStors.size()),
+                        extStors.size()
+                    ));
+                }
+
+                if (incs != null) {
+                    printer.accept("\t\t\tincremental snapshots: cnt=%d, size=%s (%db), modified='%s' (epoch=%d)".formatted(
+                        incs.number(),
+                        U.humanReadableByteCount(incs.size()),
+                        incs.size(),
+                        DATE_FORMATTER.format(Instant.ofEpochMilli(incs.date())),
+                        incs.date()
+                    ));
+                }
+            });
+        }
+
+        // Drop a line.

Review Comment:
   remove obvious (and also wrong) comment



##########
modules/core/src/test/resources/org.apache.ignite.util/GridCommandHandlerClusterByClassWithSSLTest_help.output:
##########
@@ -269,6 +269,12 @@ This utility can do the following commands:
   Get the status of the current snapshot operation:
     control.(sh|bat) --snapshot status
 
+  Lists all snapshots on all online server nodes with their sizes:

Review Comment:
   ```suggestion
     Lists all snapshots on all online server nodes, including their sizes:
   ```



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',

Review Comment:
   Probably it would be better to report about all found problems in command output ?
   I mean : if more than one node return Exception - we can report all of them and not just highlight the first found ? WDYT ?



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {

Review Comment:
   probably we need add it to utility method and also change : GridTestUtils#sizeOfDirectory, WDYT ?



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.util.UUID;
+import org.apache.ignite.internal.Order;
+import org.apache.ignite.internal.dto.IgniteDataTransferObject;
+
+/** Accumulated result of the snapshot list task. */
+public final class SnapshotListTaskResult extends IgniteDataTransferObject {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** Nodes consistent ids. */
+    @Order(0)
+    String[] cstIds;

Review Comment:
   ```suggestion
       String[] consIds;
   ```



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/CoreFeatureRegistry.java:
##########
@@ -93,4 +93,7 @@
 public class CoreFeatureRegistry {
     /** */
     public static final IgniteFeature ROLLING_UPGRADE_FEATURE = new IgniteCoreFeature(0);
+
+    /** */
+    public static final IgniteFeature SNAPSHOT_LIST_FEATURE = new IgniteCoreFeature(1);

Review Comment:
   Just a suggest - This feature will conflict with snap delete command, can you already change it for IgniteCoreFeature(2) ??



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+            // Read local snapshots.
+            List<T2<SnapshotFileTree, Long>> locSnps = findLocalSnapshots(arg.src());
+
+            if (locSnps.isEmpty())
+                return new SnapshotListJobResult(Collections.emptyMap());
+
+            // Main sizes of the local snapshots (per snapshot name).
+            Map<String, Long> sizes = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional incremental parts descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> incs = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional external storages descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> extStors = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+
+            // Snapshot sizes, incremental parts and external storages futures.
+            List<Future<?>> futs = new ArrayList<>(locSnps.size() * 3);
+            // Names of snapshots with read failures.
+            Set<String> failedSnps = ConcurrentHashMap.newKeySet(locSnps.size() / 2);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            // Descriptions of the local snapshots: main sizes, external storages and incremental parts.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                SnapshotFileTree sft = snpPair.get1();
+                String snpName = sft.name();
+
+                // Future for the snapshot main size. It is the largest part of the job, thus it goes first.
+                Future<?> fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        sizes.put(snpName, calculateDirectorySize(sft.root()));
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to calculate snapshot's size, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional incremental parts.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo incDesc = incrementals(sft);
+
+                        if (incDesc != null)
+                            incs.put(snpName, incDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's incremental parts, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional external storages.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo extDesc = externalStorages(sft);
+
+                        if (extDesc != null)
+                            extStors.put(snpName, extDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's external storages, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+            }
+
+            // Wait for the futures.
+            for (Future<?> fut : futs) {
+                try {
+                    fut.get();
+                }
+                catch (ExecutionException e) {
+                    throw new IgniteException("Failed to read local node's snapshots.", e);
+                }
+                catch (InterruptedException e) {
+                    Thread.currentThread().interrupt();
+
+                    throw new IgniteException("Interrupted while reading local snapshots.", e);
+                }
+            }
+
+            // Result snapshot descriptions.
+            Map<String, SnapshotListJobResult.SnapshotInfo> resMap = new ConcurrentHashMap<>(locSnps.size() - failedSnps.size(), 1.0f);
+
+            // Reduce results.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                String snpName = snpPair.get1().name();
+
+                if (failedSnps.contains(snpName))
+                    continue;
+
+                Long size = sizes.get(snpName);
+
+                // The size must be calculated unless the snapshot has failed.
+                assert size != null;
+
+                SnapshotListJobResult.SnapshotInfo snpDesc = new SnapshotListJobResult.SnapshotInfo(
+                    size,
+                    snpPair.get2(),
+                    extStors.get(snpName),
+                    incs.get(snpName)
+                );
+
+                resMap.put(snpName, snpDesc);
+            }
+
+            return new SnapshotListJobResult(resMap);
+        }
+
+        /** */
+        private @Nullable SnapshotListJobResult.SnapshotInfo externalStorages(SnapshotFileTree sft) {
+            int extStoragesCnt = 0;
+            long extStoragesSize = 0;
+
+            for (File extraStorage : sft.allStorages().toList()) {
+                if (sft.nodeStorage().equals(extraStorage))
+                    continue;
+
+                try {
+                    extStoragesSize += calculateDirectorySize(extraStorage);
+                }
+                catch (Exception e) {
+                    log.warning("Failed to calculate snapshot's external storage size, storage ignored [extraStorage='" +
+                        extraStorage + "']", e);
+
+                    continue;
+                }
+
+                extStoragesCnt++;
+            }
+
+            return extStoragesCnt == 0 ? null : new SnapshotListJobResult.SnapshotInfo(extStoragesCnt, extStoragesSize);
+        }
+
+        /** @return Snapshot file tree and creation time from the snapshot metadata. */
+        private List<T2<SnapshotFileTree, Long>> findLocalSnapshots(@Nullable String snpPath) {
+            // The tree is used only to extract the snapshots root directory. Such snapshot name is actually unallowed.
+            File[] dirsToParse = new SnapshotFileTree(ignite.context(), "snp", snpPath).root().getParentFile().listFiles();

Review Comment:
   "snp" - why we need such hard coded name here ?
   Seems such hardcoded name is using in `root(ft, name, path)` ?



##########
modules/control-utility/src/test/java/org/apache/ignite/util/GridCommandHandlerListSnapshotTest.java:
##########
@@ -0,0 +1,348 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.util;
+
+import java.io.File;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Collection;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.DataStorageConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommand;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.G;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.lang.IgnitePredicate;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Test;
+import org.junit.runners.Parameterized.Parameter;
+import org.junit.runners.Parameterized.Parameters;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.cluster.ClusterState.ACTIVE;
+import static org.apache.ignite.internal.commandline.CommandHandler.EXIT_CODE_OK;
+import static org.apache.ignite.internal.processors.cache.persistence.snapshot.AbstractSnapshotSelfTest.snp;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Test for the command '--snapshot list'. */
+public class GridCommandHandlerListSnapshotTest extends GridCommandHandlerAbstractTest {
+    /** External storage path. */
+    private static final String EXT_STORAGE_PATH = "extStorage";
+
+    /** Flag to use {@link DataStorageConfiguration#setExtraSnapshotPaths(String...)}. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@code extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Consistent id postfix. */
+    private @Nullable String cstIdPostfix = "";
+
+    /** Flag setting the usage of a custom snapshot path. */
+    @Parameter(1)
+    public boolean customPath;
+
+    /** Flag setting the usage of dedicated, own node working directories. */
+    @Parameter(2)
+    public boolean separatedWorkDir;
+
+    /** Flag to add external server node after the snapshot creation. */
+    @Parameter(3)
+    public boolean addExtraSrvr;
+
+    /** Number of incremental snapshots to add to the main test snapshots. */
+    @Parameter(4)
+    public int incCnt;
+
+    /** */
+    @Parameters(name = "cmdHnd={0}, customPath={1}, ownWorkDir={2}, addExtraSrvr={3}, incCnt={4}")
+    public static Collection<?> parameters() {
+        return GridTestUtils.cartesianProduct(
+            commandHandlers(),
+            F.asList(false, true), // Custom snapshot path.
+            F.asList(false, true), // Separated (own) work directories.
+            F.asList(false, true), // Add a server node after the snapshot creation.
+            F.asList(0, 2) // Number of incremental snapshots.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        super.afterTest();
+
+        stopAllGrids();
+
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTest() throws Exception {
+        super.beforeTest();
+
+        /** Handy if test running is interrupted and {@link #afterTest()} isn't invoked. */
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        // Also cleans separated snapshot working directories and custom snapshot paths.

Review Comment:
   nit: "Also" - is not acceptable here? i think it\`s copy-paste from:
   IgniteClusterSnapshotListTest#cleanPersistenceDir



##########
modules/control-utility/src/test/java/org/apache/ignite/util/GridCommandHandlerListSnapshotTest.java:
##########
@@ -0,0 +1,348 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.util;
+
+import java.io.File;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Collection;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.DataStorageConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommand;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.G;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.lang.IgnitePredicate;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Test;
+import org.junit.runners.Parameterized.Parameter;
+import org.junit.runners.Parameterized.Parameters;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.cluster.ClusterState.ACTIVE;
+import static org.apache.ignite.internal.commandline.CommandHandler.EXIT_CODE_OK;
+import static org.apache.ignite.internal.processors.cache.persistence.snapshot.AbstractSnapshotSelfTest.snp;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Test for the command '--snapshot list'. */
+public class GridCommandHandlerListSnapshotTest extends GridCommandHandlerAbstractTest {
+    /** External storage path. */
+    private static final String EXT_STORAGE_PATH = "extStorage";
+
+    /** Flag to use {@link DataStorageConfiguration#setExtraSnapshotPaths(String...)}. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@code extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Consistent id postfix. */
+    private @Nullable String cstIdPostfix = "";
+
+    /** Flag setting the usage of a custom snapshot path. */
+    @Parameter(1)
+    public boolean customPath;
+
+    /** Flag setting the usage of dedicated, own node working directories. */
+    @Parameter(2)
+    public boolean separatedWorkDir;
+
+    /** Flag to add external server node after the snapshot creation. */
+    @Parameter(3)
+    public boolean addExtraSrvr;
+
+    /** Number of incremental snapshots to add to the main test snapshots. */
+    @Parameter(4)
+    public int incCnt;
+
+    /** */
+    @Parameters(name = "cmdHnd={0}, customPath={1}, ownWorkDir={2}, addExtraSrvr={3}, incCnt={4}")
+    public static Collection<?> parameters() {
+        return GridTestUtils.cartesianProduct(
+            commandHandlers(),
+            F.asList(false, true), // Custom snapshot path.
+            F.asList(false, true), // Separated (own) work directories.
+            F.asList(false, true), // Add a server node after the snapshot creation.
+            F.asList(0, 2) // Number of incremental snapshots.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        super.afterTest();
+
+        stopAllGrids();
+
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTest() throws Exception {
+        super.beforeTest();
+
+        /** Handy if test running is interrupted and {@link #afterTest()} isn't invoked. */
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
+
+        String workDir = separatedWorkDir
+            ? new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath()
+            : U.defaultWorkDirectory();
+
+        cfg.setWorkDirectory(workDir);
+
+        cfg.getDataStorageConfiguration().setWalCompactionEnabled(incCnt > 0);
+
+        if (extStorages) {
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                workDir + File.separator,
+                workDir + File.separator + EXT_STORAGE_PATH
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", EXT_STORAGE_PATH);
+        }
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override public String getTestIgniteInstanceName() {
+        return super.getTestIgniteInstanceName() + cstIdPostfix;
+    }
+
+    /** */
+    @Test
+    public void testNoSnapshots() throws Exception {
+        doTestSnapshotsLists(0, false, false);
+    }
+
+    /** */
+    @Test
+    public void testSingleSnapshot() throws Exception {
+        doTestSnapshotsLists(1, false, false);
+    }
+
+    /** */
+    @Test
+    public void testSeveralSnapshots() throws Exception {
+        doTestSnapshotsLists(3, false, false);
+    }
+
+    /** */
+    @Test
+    public void testOneNodeMisses() throws Exception {
+        // Doesn't matter here.
+//        assumeFalse(incCnt > 0);
+        // Keeps just one node not seeing the snapshot.
+        assumeTrue(separatedWorkDir);
+
+        doTestSnapshotsLists(2, true, false);
+    }
+
+    /** */
+    @Test
+    public void testExternalStorages() throws Exception {
+        // External storages are required to be the same as configured in the node's PDS storages. Thus, we skip different work folders.
+        // Also, extra snapshot storages aren't used if snapshot is created with a custom path.
+        assumeFalse(separatedWorkDir || customPath);
+
+        extStorages = true;
+
+        doTestSnapshotsLists(2, false, false);
+    }
+
+    /** */
+    @Test
+    public void testChangedConsistentId() throws Exception {
+        // In new created working directories there will be obviously no snapshots.
+        assumeFalse(separatedWorkDir);
+
+        doTestSnapshotsLists(1, false, true);
+    }
+
+    /** */
+    private void doTestSnapshotsLists(int snpCnt, boolean deleteOnOneNode, boolean restartWithChangedCstIds) throws Exception {

Review Comment:
   You write wery hard for human reading tests ! Look you have 5 parametrized class and also input test params and common test part without detailed descriprion, if i can parse it for now - seems need huge time to parse it in future if smth. goes wrong, I suggest to cut this common test into parts of self documented ones, or if you hardly against my suggestion - add extended description.



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+            // Read local snapshots.
+            List<T2<SnapshotFileTree, Long>> locSnps = findLocalSnapshots(arg.src());
+
+            if (locSnps.isEmpty())
+                return new SnapshotListJobResult(Collections.emptyMap());
+
+            // Main sizes of the local snapshots (per snapshot name).
+            Map<String, Long> sizes = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional incremental parts descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> incs = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional external storages descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> extStors = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+
+            // Snapshot sizes, incremental parts and external storages futures.
+            List<Future<?>> futs = new ArrayList<>(locSnps.size() * 3);
+            // Names of snapshots with read failures.
+            Set<String> failedSnps = ConcurrentHashMap.newKeySet(locSnps.size() / 2);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            // Descriptions of the local snapshots: main sizes, external storages and incremental parts.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                SnapshotFileTree sft = snpPair.get1();
+                String snpName = sft.name();
+
+                // Future for the snapshot main size. It is the largest part of the job, thus it goes first.
+                Future<?> fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        sizes.put(snpName, calculateDirectorySize(sft.root()));
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to calculate snapshot's size, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional incremental parts.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo incDesc = incrementals(sft);
+
+                        if (incDesc != null)
+                            incs.put(snpName, incDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's incremental parts, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional external storages.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo extDesc = externalStorages(sft);
+
+                        if (extDesc != null)
+                            extStors.put(snpName, extDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's external storages, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+            }
+
+            // Wait for the futures.
+            for (Future<?> fut : futs) {
+                try {
+                    fut.get();
+                }
+                catch (ExecutionException e) {
+                    throw new IgniteException("Failed to read local node's snapshots.", e);
+                }
+                catch (InterruptedException e) {
+                    Thread.currentThread().interrupt();
+
+                    throw new IgniteException("Interrupted while reading local snapshots.", e);
+                }
+            }
+
+            // Result snapshot descriptions.
+            Map<String, SnapshotListJobResult.SnapshotInfo> resMap = new ConcurrentHashMap<>(locSnps.size() - failedSnps.size(), 1.0f);
+
+            // Reduce results.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                String snpName = snpPair.get1().name();
+
+                if (failedSnps.contains(snpName))
+                    continue;
+
+                Long size = sizes.get(snpName);
+
+                // The size must be calculated unless the snapshot has failed.
+                assert size != null;
+
+                SnapshotListJobResult.SnapshotInfo snpDesc = new SnapshotListJobResult.SnapshotInfo(
+                    size,
+                    snpPair.get2(),
+                    extStors.get(snpName),
+                    incs.get(snpName)
+                );
+
+                resMap.put(snpName, snpDesc);
+            }
+
+            return new SnapshotListJobResult(resMap);
+        }
+
+        /** */
+        private @Nullable SnapshotListJobResult.SnapshotInfo externalStorages(SnapshotFileTree sft) {
+            int extStoragesCnt = 0;
+            long extStoragesSize = 0;
+
+            for (File extraStorage : sft.allStorages().toList()) {
+                if (sft.nodeStorage().equals(extraStorage))
+                    continue;
+
+                try {
+                    extStoragesSize += calculateDirectorySize(extraStorage);
+                }
+                catch (Exception e) {
+                    log.warning("Failed to calculate snapshot's external storage size, storage ignored [extraStorage='" +
+                        extraStorage + "']", e);
+
+                    continue;
+                }
+
+                extStoragesCnt++;
+            }
+
+            return extStoragesCnt == 0 ? null : new SnapshotListJobResult.SnapshotInfo(extStoragesCnt, extStoragesSize);
+        }
+
+        /** @return Snapshot file tree and creation time from the snapshot metadata. */
+        private List<T2<SnapshotFileTree, Long>> findLocalSnapshots(@Nullable String snpPath) {
+            // The tree is used only to extract the snapshots root directory. Such snapshot name is actually unallowed.
+            File[] dirsToParse = new SnapshotFileTree(ignite.context(), "snp", snpPath).root().getParentFile().listFiles();
+
+            if (F.isEmpty(dirsToParse))
+                return Collections.emptyList();
+
+            List<Future<T2<SnapshotFileTree, Long>>> futs = new ArrayList<>(dirsToParse.length);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            for (File snpDir : dirsToParse) {
+                Future<T2<SnapshotFileTree, Long>> snpDirFut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    String snpName = snpDir.getName();
+
+                    // Snapshot tree being used as a path, to read the metas only.
+                    SnapshotFileTree sft = new SnapshotFileTree(ignite.context(), snpName, snpPath);
+
+                    List<SnapshotMetadata> metas = ignite.context().cache().context().snapshotMgr().readSnapshotMetadatas(sft, false);
+
+                    if (metas.isEmpty())
+                        return null;
+
+                    // Get the last-created time snapshot metadata.
+                    SnapshotMetadata snpMeta = metas.stream().max(Comparator.comparingLong(SnapshotMetadata::snapshotTime)).get();
+
+                    // Real, meta-based snapshot file tree. Can belong to other cluster, other consistent id.

Review Comment:
   Miss this comment, plz clarify it ?



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";

Review Comment:
   ```suggestion
       public static final String HEADER = "Snapshots were found on the following nodes:";
   ```



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");

Review Comment:
   1. No need to comment such obvious code, + you write "skip" but this is not a skip but appending 
   2. I prefer to remove this code at all, seems all ok without such division ? 
   ```
   Snapshots found on the following nodes:
   	Node 'gridCommandHandlerTest0' [uuid=dadfbb1c-8a19-48bb-9ccd-ecf5a9000000]:
   		Snapshot 'testSnapshot0': totalSize=671.0 KB (687131b), created='2026-10-19 09:50:10 +0300' (epoch=1792392610063)
   	Node 'gridCommandHandlerTest1' [uuid=ee213f2a-7257-42f7-ae37-61f65c600001]:
   		Snapshot 'testSnapshot0': totalSize=671.0 KB (687131b), created='2026-10-19 09:50:10 +0300' (epoch=1792392610063)
   	Node 'gridCommandHandlerTest2' [uuid=7769f8bc-f29f-40e2-83bd-33ed36c00002]:
   		Snapshot 'testSnapshot0': totalSize=671.0 KB (687131b), created='2026-10-19 09:50:10 +0300' (epoch=1792392610063)
   ```



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";

Review Comment:
   ```suggestion
       public static final String DESC = "Lists snapshots on all available server nodes";
   ```



##########
modules/core/src/test/resources/org.apache.ignite.util/GridCommandHandlerClusterByClassTest_help.output:
##########
@@ -269,6 +269,12 @@ This utility can do the following commands:
   Get the status of the current snapshot operation:
     control.(sh|bat) --snapshot status
 
+  Lists all snapshots on all online server nodes with their sizes:

Review Comment:
   ```suggestion
    Lists all snapshots on all online server nodes, including their sizes:
   ```



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {

Review Comment:
   Probably it need to be a utility method ? You already need it in snap delete command.



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");
+
+            printer.accept("\tNode '%s' [uuid=%s]:".formatted(res.consistentIds()[nodeIdx], res.nodesIds()[nodeIdx]));
+
+            SnapshotListJobResult nodeResult = res.nodesSnapshots()[nodeIdx];
+
+            if (nodeResult.snapshots().isEmpty()) {
+                printer.accept("\t\t" + NO_SNAPSHOTS);
+
+                continue;
+            }
+
+            // Sorted snapshots names for better user reading.
+            new TreeMap<>(nodeResult.snapshots()).forEach((snpName, snpInfo) -> {
+                printer.accept("\t\tSnapshot '%s': totalSize=%s (%db), created='%s' (epoch=%d)".formatted(

Review Comment:
   Output like : `Snapshot 'testSnapshot0': totalSize=659.0 KB (674843b), created='2026-10-19 09:11:00 +0300' (epoch=1792390260252)`
   Why do you deside that:  1. **size in bytes** 2. **epoch** is helpful somehow ? I just think it looks like a garbage, fix me if i\`m wrong.



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));

Review Comment:
   I think this is not informative `cause` and you need to use : IgniteException(java.lang.String) instead ?



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+            // Read local snapshots.
+            List<T2<SnapshotFileTree, Long>> locSnps = findLocalSnapshots(arg.src());
+
+            if (locSnps.isEmpty())
+                return new SnapshotListJobResult(Collections.emptyMap());
+
+            // Main sizes of the local snapshots (per snapshot name).
+            Map<String, Long> sizes = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional incremental parts descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> incs = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional external storages descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> extStors = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+
+            // Snapshot sizes, incremental parts and external storages futures.
+            List<Future<?>> futs = new ArrayList<>(locSnps.size() * 3);
+            // Names of snapshots with read failures.
+            Set<String> failedSnps = ConcurrentHashMap.newKeySet(locSnps.size() / 2);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            // Descriptions of the local snapshots: main sizes, external storages and incremental parts.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                SnapshotFileTree sft = snpPair.get1();
+                String snpName = sft.name();
+
+                // Future for the snapshot main size. It is the largest part of the job, thus it goes first.
+                Future<?> fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));

Review Comment:
   t he same as above



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+            // Read local snapshots.
+            List<T2<SnapshotFileTree, Long>> locSnps = findLocalSnapshots(arg.src());
+
+            if (locSnps.isEmpty())
+                return new SnapshotListJobResult(Collections.emptyMap());
+
+            // Main sizes of the local snapshots (per snapshot name).
+            Map<String, Long> sizes = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional incremental parts descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> incs = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional external storages descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> extStors = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+
+            // Snapshot sizes, incremental parts and external storages futures.
+            List<Future<?>> futs = new ArrayList<>(locSnps.size() * 3);
+            // Names of snapshots with read failures.
+            Set<String> failedSnps = ConcurrentHashMap.newKeySet(locSnps.size() / 2);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            // Descriptions of the local snapshots: main sizes, external storages and incremental parts.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                SnapshotFileTree sft = snpPair.get1();
+                String snpName = sft.name();
+
+                // Future for the snapshot main size. It is the largest part of the job, thus it goes first.
+                Future<?> fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        sizes.put(snpName, calculateDirectorySize(sft.root()));
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to calculate snapshot's size, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional incremental parts.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo incDesc = incrementals(sft);
+
+                        if (incDesc != null)
+                            incs.put(snpName, incDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's incremental parts, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional external storages.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo extDesc = externalStorages(sft);
+
+                        if (extDesc != null)
+                            extStors.put(snpName, extDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's external storages, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+            }
+
+            // Wait for the futures.
+            for (Future<?> fut : futs) {
+                try {
+                    fut.get();
+                }
+                catch (ExecutionException e) {
+                    throw new IgniteException("Failed to read local node's snapshots.", e);
+                }
+                catch (InterruptedException e) {
+                    Thread.currentThread().interrupt();
+
+                    throw new IgniteException("Interrupted while reading local snapshots.", e);
+                }
+            }
+
+            // Result snapshot descriptions.
+            Map<String, SnapshotListJobResult.SnapshotInfo> resMap = new ConcurrentHashMap<>(locSnps.size() - failedSnps.size(), 1.0f);

Review Comment:
   why concMap here ?



##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListTest.java:
##########
@@ -0,0 +1,936 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.io.Serializable;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Stream;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.lang.IgniteFuture;
+import org.apache.ignite.plugin.AbstractTestPluginProvider;
+import org.apache.ignite.plugin.PluginConfiguration;
+import org.apache.ignite.plugin.PluginContext;
+import org.apache.ignite.plugin.PluginProvider;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runners.Parameterized;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.configuration.IgniteConfiguration.DFLT_SNAPSHOT_THREAD_POOL_SIZE;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.cartesianProduct;
+import static org.apache.ignite.testframework.GridTestUtils.runAsync;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Cluster-wide snapshot list procedure tests. */
+public class IgniteClusterSnapshotListTest extends AbstractSnapshotSelfTest {
+    /** Number of cache keys to pre-create at node startup. */
+    private static final int CACHE_KEYS_RANGE = 15;
+
+    /** Number of partitions of the test cache. */
+    private static final int CACHE_PARTITIONS_COUNT = 4;
+
+    /** Flag showing that the filesystem supports POSIX permissions. */
+    private static boolean posixPermissions;
+
+    /** Size of the snapshot utility thread pool. */
+    @Parameterized.Parameter(2)
+    public int snpThrdPoolSz;
+
+    /** */
+    private PluginProvider<PluginConfiguration> pluginProvider;
+
+    /** Flag to spread the test cache data over the external storages. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@link #extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Parameters. */
+    @Parameterized.Parameters(name = "encryption={0}, onlyPrimary={1}, snpThrdPoolSz={2}")
+    public static Collection<Object[]> params() {
+        return cartesianProduct(
+            encryptionParameters(), // Encryption.
+            F.asList(false, true), // Only primary.
+            F.asList(DFLT_SNAPSHOT_THREAD_POOL_SIZE, 1) // Snapshots thread pool size.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName)
+            .setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        if (pluginProvider != null)
+            cfg.setPluginProviders(pluginProvider);
+
+        if (extStorages) {
+            // External storage paths must be identical on all the nodes (a cache has the single storage paths
+            // setting). Thus, a shared work directory is used, like in GridCommandHandlerListSnapshotTest.
+            cfg.setWorkDirectory(U.defaultWorkDirectory());
+
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                U.defaultWorkDirectory() + File.separator,
+                U.defaultWorkDirectory() + File.separator + "extStorage"
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", "extStorage");
+        }
+
+        cfg.setSnapshotThreadPoolSize(snpThrdPoolSz);
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        super.cleanPersistenceDir();
+
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTestsStarted() throws Exception {
+        super.beforeTestsStarted();
+
+        File workDir = new File(U.defaultWorkDirectory());
+
+        assertTrue(workDir.exists());
+
+        try {
+            Files.getPosixFilePermissions(workDir.toPath());
+
+            posixPermissions = true;
+        }
+        catch (UnsupportedOperationException ignored) {
+            // No-op.
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected <K, V> CacheConfiguration<K, V> txCacheConfig(CacheConfiguration<K, V> ccfg) {
+        // Speeds up the tests.
+        ccfg = super.txCacheConfig(ccfg)
+            .setAffinity(new RendezvousAffinityFunction(false, CACHE_PARTITIONS_COUNT))
+            .setBackups(1);
+
+        if (extStorages) {
+            assert !F.isEmpty(extStoragePaths);
+
+            ccfg.setStoragePaths(extStoragePaths);
+        }
+
+        return ccfg;
+    }
+
+    /** */
+    @Test
+    public void testConcurrentCreation() throws Exception {
+        // The test uses the thread blocking and conditional waiting. Won't proceed with 1 thread.
+        assumeTrue(snpThrdPoolSz > 1);
+
+        int grids = 3;
+        int testNodeIdx = 1;
+        int testNodeOrder = testNodeIdx + 1;
+
+        CountDownLatch beginSnpCreation = new CountDownLatch(grids);
+        CountDownLatch proceedSnpCreation = new CountDownLatch(1);
+
+        // Delays snapshot creation after its metadata is written.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public <M extends Serializable> void storeSnapshotMeta(M meta, File smf) {
+                            super.storeSnapshotMeta(meta, smf);
+
+                            beginSnpCreation.countDown();
+
+                            if (((IgniteEx)ctx.grid()).localNode().order() == testNodeOrder) {
+                                try {
+                                    assertTrue(proceedSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        IgniteFuture<Void> createSnpFut = snp(grid(0)).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary);
+
+        assertTrue(beginSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        SnapshotListTaskResult lstOpRes = listSnapshots(grid(0));
+
+        int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+        assertEquals(grids, snpsCnt);
+
+        SnapshotFileTree testSft = new SnapshotFileTree(grid(testNodeIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(SnapshotListTask.calculateDirectorySize(testSft.root()) > 0L);
+
+        proceedSnpCreation.countDown();
+
+        createSnpFut.get(getTestTimeout());
+    }
+
+    /** */
+    @Test
+    public void testCompletelyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(true);
+    }
+
+    /** */
+    @Test
+    public void testPartlyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(false);
+    }
+
+    /** */
+    private void doTestDeletionAfterMetaRead(boolean completeDeletion) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        CountDownLatch metaReadBeginLatch = new CountDownLatch(grids);
+        CountDownLatch metaReadProceedLatch = new CountDownLatch(1);
+
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft, boolean failIfCantRead) {
+                            metaReadBeginLatch.countDown();
+
+                            if (cctx.localNode().order() == testGridIdx + 1) {
+                                try {
+                                    assertTrue(metaReadProceedLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        long testSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+        assertTrue(testSnpSz > 0);
+
+        IgniteInternalFuture<SnapshotListTaskResult> lstOpFut = runAsync(() -> listSnapshots(grid(0)));
+
+        assertTrue(metaReadBeginLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        if (completeDeletion)
+            assertTrue(testSnpFt.root().exists() && U.delete(testSnpFt.root()) && !testSnpFt.root().exists());
+        else
+            assertTrue(testSnpFt.nodeStorage().exists() && U.delete(testSnpFt.nodeStorage()) && !testSnpFt.nodeStorage().exists());
+
+        metaReadProceedLatch.countDown();
+
+        SnapshotListTaskResult lstOpRes = lstOpFut.get();
+
+        if (completeDeletion) {
+            int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+            assertEquals(grids - 1, snpsCnt);
+        }
+        else {
+            int snpsCnt = 0;
+            boolean victimNodeFound = false;
+
+            for (int i = 0; i < lstOpRes.nodesIds().length; i++) {
+                UUID nid = lstOpRes.nodesIds()[i];
+
+                Map<String, SnapshotListJobResult.SnapshotInfo> nodeSnps = lstOpRes.nodesSnapshots()[i].snapshots();
+
+                snpsCnt += nodeSnps.size();
+
+                if (nid.equals(grid(testGridIdx).localNode().id())) {
+                    victimNodeFound = true;
+
+                    assertEquals(1, nodeSnps.size());
+
+                    long curSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+                    assertTrue(curSnpSz < testSnpSz);
+                }
+            }
+
+            assertTrue(victimNodeFound);
+            assertEquals(grids, snpsCnt);
+        }
+    }
+
+    /** */
+    @Test
+    public void testMissingSnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(true);
+    }
+
+    /** */
+    @Test
+    public void testEmptySnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(false);
+    }
+
+    /** */
+    private void doTestWrongSnapshotPath(boolean missing) throws Exception {
+        // Doesn't matter here, speeds up the tests.
+        assumeFalse(snpThrdPoolSz < 2);
+
+        doTestSnapshotPath(null);

Review Comment:
   Also clarify - why you need to create a snap if you use different Directory ?



##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+            // Read local snapshots.
+            List<T2<SnapshotFileTree, Long>> locSnps = findLocalSnapshots(arg.src());
+
+            if (locSnps.isEmpty())
+                return new SnapshotListJobResult(Collections.emptyMap());
+
+            // Main sizes of the local snapshots (per snapshot name).
+            Map<String, Long> sizes = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional incremental parts descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> incs = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional external storages descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> extStors = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+
+            // Snapshot sizes, incremental parts and external storages futures.
+            List<Future<?>> futs = new ArrayList<>(locSnps.size() * 3);
+            // Names of snapshots with read failures.
+            Set<String> failedSnps = ConcurrentHashMap.newKeySet(locSnps.size() / 2);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            // Descriptions of the local snapshots: main sizes, external storages and incremental parts.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                SnapshotFileTree sft = snpPair.get1();
+                String snpName = sft.name();
+
+                // Future for the snapshot main size. It is the largest part of the job, thus it goes first.
+                Future<?> fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        sizes.put(snpName, calculateDirectorySize(sft.root()));
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to calculate snapshot's size, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional incremental parts.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo incDesc = incrementals(sft);
+
+                        if (incDesc != null)
+                            incs.put(snpName, incDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's incremental parts, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional external storages.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo extDesc = externalStorages(sft);
+
+                        if (extDesc != null)
+                            extStors.put(snpName, extDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's external storages, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+            }
+
+            // Wait for the futures.
+            for (Future<?> fut : futs) {
+                try {
+                    fut.get();
+                }
+                catch (ExecutionException e) {
+                    throw new IgniteException("Failed to read local node's snapshots.", e);
+                }
+                catch (InterruptedException e) {
+                    Thread.currentThread().interrupt();
+
+                    throw new IgniteException("Interrupted while reading local snapshots.", e);
+                }
+            }
+
+            // Result snapshot descriptions.
+            Map<String, SnapshotListJobResult.SnapshotInfo> resMap = new ConcurrentHashMap<>(locSnps.size() - failedSnps.size(), 1.0f);
+
+            // Reduce results.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                String snpName = snpPair.get1().name();
+
+                if (failedSnps.contains(snpName))
+                    continue;
+
+                Long size = sizes.get(snpName);
+
+                // The size must be calculated unless the snapshot has failed.
+                assert size != null;
+
+                SnapshotListJobResult.SnapshotInfo snpDesc = new SnapshotListJobResult.SnapshotInfo(
+                    size,
+                    snpPair.get2(),
+                    extStors.get(snpName),
+                    incs.get(snpName)
+                );
+
+                resMap.put(snpName, snpDesc);
+            }
+
+            return new SnapshotListJobResult(resMap);
+        }
+
+        /** */
+        private @Nullable SnapshotListJobResult.SnapshotInfo externalStorages(SnapshotFileTree sft) {
+            int extStoragesCnt = 0;
+            long extStoragesSize = 0;
+
+            for (File extraStorage : sft.allStorages().toList()) {
+                if (sft.nodeStorage().equals(extraStorage))
+                    continue;
+
+                try {
+                    extStoragesSize += calculateDirectorySize(extraStorage);
+                }
+                catch (Exception e) {
+                    log.warning("Failed to calculate snapshot's external storage size, storage ignored [extraStorage='" +
+                        extraStorage + "']", e);
+
+                    continue;
+                }
+
+                extStoragesCnt++;
+            }
+
+            return extStoragesCnt == 0 ? null : new SnapshotListJobResult.SnapshotInfo(extStoragesCnt, extStoragesSize);
+        }
+
+        /** @return Snapshot file tree and creation time from the snapshot metadata. */
+        private List<T2<SnapshotFileTree, Long>> findLocalSnapshots(@Nullable String snpPath) {
+            // The tree is used only to extract the snapshots root directory. Such snapshot name is actually unallowed.
+            File[] dirsToParse = new SnapshotFileTree(ignite.context(), "snp", snpPath).root().getParentFile().listFiles();
+
+            if (F.isEmpty(dirsToParse))
+                return Collections.emptyList();
+
+            List<Future<T2<SnapshotFileTree, Long>>> futs = new ArrayList<>(dirsToParse.length);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();

Review Comment:
   Why u decide to parallel IO operations, is it really helpful ? Do you guarantee no spikes are raised during concurrent cache\tx operations in such a case ?



##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListTest.java:
##########
@@ -0,0 +1,936 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.io.Serializable;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Stream;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.lang.IgniteFuture;
+import org.apache.ignite.plugin.AbstractTestPluginProvider;
+import org.apache.ignite.plugin.PluginConfiguration;
+import org.apache.ignite.plugin.PluginContext;
+import org.apache.ignite.plugin.PluginProvider;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runners.Parameterized;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.configuration.IgniteConfiguration.DFLT_SNAPSHOT_THREAD_POOL_SIZE;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.cartesianProduct;
+import static org.apache.ignite.testframework.GridTestUtils.runAsync;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Cluster-wide snapshot list procedure tests. */
+public class IgniteClusterSnapshotListTest extends AbstractSnapshotSelfTest {
+    /** Number of cache keys to pre-create at node startup. */
+    private static final int CACHE_KEYS_RANGE = 15;
+
+    /** Number of partitions of the test cache. */
+    private static final int CACHE_PARTITIONS_COUNT = 4;
+
+    /** Flag showing that the filesystem supports POSIX permissions. */
+    private static boolean posixPermissions;
+
+    /** Size of the snapshot utility thread pool. */
+    @Parameterized.Parameter(2)
+    public int snpThrdPoolSz;
+
+    /** */
+    private PluginProvider<PluginConfiguration> pluginProvider;
+
+    /** Flag to spread the test cache data over the external storages. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@link #extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Parameters. */
+    @Parameterized.Parameters(name = "encryption={0}, onlyPrimary={1}, snpThrdPoolSz={2}")
+    public static Collection<Object[]> params() {
+        return cartesianProduct(
+            encryptionParameters(), // Encryption.
+            F.asList(false, true), // Only primary.
+            F.asList(DFLT_SNAPSHOT_THREAD_POOL_SIZE, 1) // Snapshots thread pool size.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName)
+            .setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        if (pluginProvider != null)
+            cfg.setPluginProviders(pluginProvider);
+
+        if (extStorages) {
+            // External storage paths must be identical on all the nodes (a cache has the single storage paths
+            // setting). Thus, a shared work directory is used, like in GridCommandHandlerListSnapshotTest.
+            cfg.setWorkDirectory(U.defaultWorkDirectory());
+
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                U.defaultWorkDirectory() + File.separator,
+                U.defaultWorkDirectory() + File.separator + "extStorage"
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", "extStorage");
+        }
+
+        cfg.setSnapshotThreadPoolSize(snpThrdPoolSz);
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        super.cleanPersistenceDir();
+
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTestsStarted() throws Exception {
+        super.beforeTestsStarted();
+
+        File workDir = new File(U.defaultWorkDirectory());
+
+        assertTrue(workDir.exists());
+
+        try {
+            Files.getPosixFilePermissions(workDir.toPath());
+
+            posixPermissions = true;
+        }
+        catch (UnsupportedOperationException ignored) {
+            // No-op.
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected <K, V> CacheConfiguration<K, V> txCacheConfig(CacheConfiguration<K, V> ccfg) {
+        // Speeds up the tests.
+        ccfg = super.txCacheConfig(ccfg)
+            .setAffinity(new RendezvousAffinityFunction(false, CACHE_PARTITIONS_COUNT))
+            .setBackups(1);
+
+        if (extStorages) {
+            assert !F.isEmpty(extStoragePaths);
+
+            ccfg.setStoragePaths(extStoragePaths);
+        }
+
+        return ccfg;
+    }
+
+    /** */
+    @Test
+    public void testConcurrentCreation() throws Exception {
+        // The test uses the thread blocking and conditional waiting. Won't proceed with 1 thread.
+        assumeTrue(snpThrdPoolSz > 1);
+
+        int grids = 3;
+        int testNodeIdx = 1;
+        int testNodeOrder = testNodeIdx + 1;
+
+        CountDownLatch beginSnpCreation = new CountDownLatch(grids);
+        CountDownLatch proceedSnpCreation = new CountDownLatch(1);
+
+        // Delays snapshot creation after its metadata is written.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public <M extends Serializable> void storeSnapshotMeta(M meta, File smf) {
+                            super.storeSnapshotMeta(meta, smf);
+
+                            beginSnpCreation.countDown();
+
+                            if (((IgniteEx)ctx.grid()).localNode().order() == testNodeOrder) {
+                                try {
+                                    assertTrue(proceedSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        IgniteFuture<Void> createSnpFut = snp(grid(0)).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary);
+
+        assertTrue(beginSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        SnapshotListTaskResult lstOpRes = listSnapshots(grid(0));
+
+        int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+        assertEquals(grids, snpsCnt);
+
+        SnapshotFileTree testSft = new SnapshotFileTree(grid(testNodeIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(SnapshotListTask.calculateDirectorySize(testSft.root()) > 0L);
+
+        proceedSnpCreation.countDown();
+
+        createSnpFut.get(getTestTimeout());
+    }
+
+    /** */
+    @Test
+    public void testCompletelyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(true);
+    }
+
+    /** */
+    @Test
+    public void testPartlyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(false);
+    }
+
+    /** */
+    private void doTestDeletionAfterMetaRead(boolean completeDeletion) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        CountDownLatch metaReadBeginLatch = new CountDownLatch(grids);
+        CountDownLatch metaReadProceedLatch = new CountDownLatch(1);
+
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft, boolean failIfCantRead) {
+                            metaReadBeginLatch.countDown();
+
+                            if (cctx.localNode().order() == testGridIdx + 1) {
+                                try {
+                                    assertTrue(metaReadProceedLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        long testSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+        assertTrue(testSnpSz > 0);
+
+        IgniteInternalFuture<SnapshotListTaskResult> lstOpFut = runAsync(() -> listSnapshots(grid(0)));
+
+        assertTrue(metaReadBeginLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        if (completeDeletion)
+            assertTrue(testSnpFt.root().exists() && U.delete(testSnpFt.root()) && !testSnpFt.root().exists());
+        else
+            assertTrue(testSnpFt.nodeStorage().exists() && U.delete(testSnpFt.nodeStorage()) && !testSnpFt.nodeStorage().exists());
+
+        metaReadProceedLatch.countDown();
+
+        SnapshotListTaskResult lstOpRes = lstOpFut.get();
+
+        if (completeDeletion) {
+            int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+            assertEquals(grids - 1, snpsCnt);
+        }
+        else {
+            int snpsCnt = 0;
+            boolean victimNodeFound = false;
+
+            for (int i = 0; i < lstOpRes.nodesIds().length; i++) {
+                UUID nid = lstOpRes.nodesIds()[i];
+
+                Map<String, SnapshotListJobResult.SnapshotInfo> nodeSnps = lstOpRes.nodesSnapshots()[i].snapshots();
+
+                snpsCnt += nodeSnps.size();
+
+                if (nid.equals(grid(testGridIdx).localNode().id())) {
+                    victimNodeFound = true;
+
+                    assertEquals(1, nodeSnps.size());
+
+                    long curSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+                    assertTrue(curSnpSz < testSnpSz);
+                }
+            }
+
+            assertTrue(victimNodeFound);
+            assertEquals(grids, snpsCnt);
+        }
+    }
+
+    /** */
+    @Test
+    public void testMissingSnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(true);
+    }
+
+    /** */
+    @Test
+    public void testEmptySnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(false);
+    }
+
+    /** */
+    private void doTestWrongSnapshotPath(boolean missing) throws Exception {
+        // Doesn't matter here, speeds up the tests.
+        assumeFalse(snpThrdPoolSz < 2);

Review Comment:
   ```suggestion
       private void doTestWrongSnapshotPath(boolean missing) throws Exception {
           // Doesn't matter here, speeds up the tests.
           snpThrdPoolSz = DFLT_SNAPSHOT_THREAD_POOL_SIZE;
   
           assumeFalse(snpThrdPoolSz < 2);
   ```
   Seems you need to simplify tests, remove `snpThrdPoolSz` from params and use directly



##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListTest.java:
##########
@@ -0,0 +1,936 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.io.Serializable;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Stream;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.lang.IgniteFuture;
+import org.apache.ignite.plugin.AbstractTestPluginProvider;
+import org.apache.ignite.plugin.PluginConfiguration;
+import org.apache.ignite.plugin.PluginContext;
+import org.apache.ignite.plugin.PluginProvider;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runners.Parameterized;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.configuration.IgniteConfiguration.DFLT_SNAPSHOT_THREAD_POOL_SIZE;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.cartesianProduct;
+import static org.apache.ignite.testframework.GridTestUtils.runAsync;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Cluster-wide snapshot list procedure tests. */
+public class IgniteClusterSnapshotListTest extends AbstractSnapshotSelfTest {
+    /** Number of cache keys to pre-create at node startup. */
+    private static final int CACHE_KEYS_RANGE = 15;
+
+    /** Number of partitions of the test cache. */
+    private static final int CACHE_PARTITIONS_COUNT = 4;
+
+    /** Flag showing that the filesystem supports POSIX permissions. */
+    private static boolean posixPermissions;
+
+    /** Size of the snapshot utility thread pool. */
+    @Parameterized.Parameter(2)
+    public int snpThrdPoolSz;
+
+    /** */
+    private PluginProvider<PluginConfiguration> pluginProvider;
+
+    /** Flag to spread the test cache data over the external storages. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@link #extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Parameters. */
+    @Parameterized.Parameters(name = "encryption={0}, onlyPrimary={1}, snpThrdPoolSz={2}")
+    public static Collection<Object[]> params() {
+        return cartesianProduct(
+            encryptionParameters(), // Encryption.
+            F.asList(false, true), // Only primary.
+            F.asList(DFLT_SNAPSHOT_THREAD_POOL_SIZE, 1) // Snapshots thread pool size.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName)
+            .setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        if (pluginProvider != null)
+            cfg.setPluginProviders(pluginProvider);
+
+        if (extStorages) {
+            // External storage paths must be identical on all the nodes (a cache has the single storage paths
+            // setting). Thus, a shared work directory is used, like in GridCommandHandlerListSnapshotTest.
+            cfg.setWorkDirectory(U.defaultWorkDirectory());
+
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                U.defaultWorkDirectory() + File.separator,
+                U.defaultWorkDirectory() + File.separator + "extStorage"
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", "extStorage");
+        }
+
+        cfg.setSnapshotThreadPoolSize(snpThrdPoolSz);
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        super.cleanPersistenceDir();
+
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTestsStarted() throws Exception {
+        super.beforeTestsStarted();
+
+        File workDir = new File(U.defaultWorkDirectory());
+
+        assertTrue(workDir.exists());
+
+        try {
+            Files.getPosixFilePermissions(workDir.toPath());
+
+            posixPermissions = true;
+        }
+        catch (UnsupportedOperationException ignored) {
+            // No-op.
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected <K, V> CacheConfiguration<K, V> txCacheConfig(CacheConfiguration<K, V> ccfg) {
+        // Speeds up the tests.
+        ccfg = super.txCacheConfig(ccfg)
+            .setAffinity(new RendezvousAffinityFunction(false, CACHE_PARTITIONS_COUNT))
+            .setBackups(1);
+
+        if (extStorages) {
+            assert !F.isEmpty(extStoragePaths);
+
+            ccfg.setStoragePaths(extStoragePaths);
+        }
+
+        return ccfg;
+    }
+
+    /** */
+    @Test
+    public void testConcurrentCreation() throws Exception {
+        // The test uses the thread blocking and conditional waiting. Won't proceed with 1 thread.
+        assumeTrue(snpThrdPoolSz > 1);
+
+        int grids = 3;
+        int testNodeIdx = 1;
+        int testNodeOrder = testNodeIdx + 1;
+
+        CountDownLatch beginSnpCreation = new CountDownLatch(grids);
+        CountDownLatch proceedSnpCreation = new CountDownLatch(1);
+
+        // Delays snapshot creation after its metadata is written.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public <M extends Serializable> void storeSnapshotMeta(M meta, File smf) {
+                            super.storeSnapshotMeta(meta, smf);
+
+                            beginSnpCreation.countDown();
+
+                            if (((IgniteEx)ctx.grid()).localNode().order() == testNodeOrder) {
+                                try {
+                                    assertTrue(proceedSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        IgniteFuture<Void> createSnpFut = snp(grid(0)).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary);
+
+        assertTrue(beginSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        SnapshotListTaskResult lstOpRes = listSnapshots(grid(0));
+
+        int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+        assertEquals(grids, snpsCnt);
+
+        SnapshotFileTree testSft = new SnapshotFileTree(grid(testNodeIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(SnapshotListTask.calculateDirectorySize(testSft.root()) > 0L);
+
+        proceedSnpCreation.countDown();
+
+        createSnpFut.get(getTestTimeout());
+    }
+
+    /** */
+    @Test
+    public void testCompletelyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(true);
+    }
+
+    /** */
+    @Test
+    public void testPartlyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(false);
+    }
+
+    /** */
+    private void doTestDeletionAfterMetaRead(boolean completeDeletion) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        CountDownLatch metaReadBeginLatch = new CountDownLatch(grids);
+        CountDownLatch metaReadProceedLatch = new CountDownLatch(1);
+
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft, boolean failIfCantRead) {
+                            metaReadBeginLatch.countDown();
+
+                            if (cctx.localNode().order() == testGridIdx + 1) {
+                                try {
+                                    assertTrue(metaReadProceedLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        long testSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+        assertTrue(testSnpSz > 0);
+
+        IgniteInternalFuture<SnapshotListTaskResult> lstOpFut = runAsync(() -> listSnapshots(grid(0)));
+
+        assertTrue(metaReadBeginLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        if (completeDeletion)
+            assertTrue(testSnpFt.root().exists() && U.delete(testSnpFt.root()) && !testSnpFt.root().exists());
+        else
+            assertTrue(testSnpFt.nodeStorage().exists() && U.delete(testSnpFt.nodeStorage()) && !testSnpFt.nodeStorage().exists());
+
+        metaReadProceedLatch.countDown();
+
+        SnapshotListTaskResult lstOpRes = lstOpFut.get();
+
+        if (completeDeletion) {
+            int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+            assertEquals(grids - 1, snpsCnt);
+        }
+        else {
+            int snpsCnt = 0;
+            boolean victimNodeFound = false;
+
+            for (int i = 0; i < lstOpRes.nodesIds().length; i++) {
+                UUID nid = lstOpRes.nodesIds()[i];
+
+                Map<String, SnapshotListJobResult.SnapshotInfo> nodeSnps = lstOpRes.nodesSnapshots()[i].snapshots();
+
+                snpsCnt += nodeSnps.size();
+
+                if (nid.equals(grid(testGridIdx).localNode().id())) {
+                    victimNodeFound = true;
+
+                    assertEquals(1, nodeSnps.size());
+
+                    long curSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+                    assertTrue(curSnpSz < testSnpSz);
+                }
+            }
+
+            assertTrue(victimNodeFound);
+            assertEquals(grids, snpsCnt);
+        }
+    }
+
+    /** */
+    @Test
+    public void testMissingSnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(true);
+    }
+
+    /** */
+    @Test
+    public void testEmptySnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(false);
+    }
+
+    /** */
+    private void doTestWrongSnapshotPath(boolean missing) throws Exception {
+        // Doesn't matter here, speeds up the tests.
+        assumeFalse(snpThrdPoolSz < 2);
+
+        doTestSnapshotPath(null);
+
+        File path = new File(U.defaultWorkDirectory(), "not_snapshots");
+
+        if (!missing)
+            assertTrue(new File(path, SNAPSHOT_NAME).mkdirs());
+
+        SnapshotListJobResult[] lstOpRes = listSnapshots(grid(0), path.getAbsolutePath()).nodesSnapshots();
+
+        int cnt = Stream.of(lstOpRes).mapToInt(nodeRes -> nodeRes.snapshots().size()).sum();
+
+        assertEquals(0, cnt);
+    }
+
+    /** */
+    @Test
+    public void testDefaultSnapshotPath() throws Exception {
+        doTestSnapshotPath(null);
+    }
+
+    /** */
+    @Test
+    @Ignore("https://issues.apache.org/jira/browse/IGNITE-29126")
+    public void testRelativeSnapshotPath() throws Exception {
+        doTestSnapshotPath("ex_snapshots");
+    }
+
+    /** */
+    @Test
+    public void testAbsoluteSnapshotPath() throws Exception {
+        doTestSnapshotPath(new File(U.defaultWorkDirectory(), "ex_snapshots").getAbsolutePath());
+    }
+
+    /** */
+    private void doTestSnapshotPath(@Nullable String path) throws Exception {
+        int grids = 3;
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        snp(grid(0)).createSnapshot(SNAPSHOT_NAME, path, false, onlyPrimary).get(getTestTimeout());
+
+        SnapshotListJobResult[] lstOpRes = listSnapshots(grid(0), path).nodesSnapshots();
+
+        int cnt = Stream.of(lstOpRes).mapToInt(nodeRes -> nodeRes.snapshots().size()).sum();
+
+        assertEquals(grids, cnt);
+    }
+
+    /** */
+    @Test
+    public void testMissingMeta() throws Exception {
+        doTestWithWrongMeta(false);
+    }
+
+    /** */
+    @Test
+    public void testCorruptedMeta() throws Exception {
+        doTestWithWrongMeta(true);
+    }
+
+    /**
+     * Tests snapshot lists when one snapshot metadata cannot be read.
+     *
+     * @param corruptFile If {@code true}, corrupts metadata. Otherwise, deletes metadata.
+     */
+    private void doTestWithWrongMeta(boolean corruptFile) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(testSnpFt.meta().exists());
+
+        if (corruptFile) {
+            try (RandomAccessFile raf = new RandomAccessFile(testSnpFt.meta(), "rw")) {
+                raf.write(UUID.randomUUID().toString().getBytes());
+            }
+        }
+        else
+            assertTrue(testSnpFt.meta().delete() && !testSnpFt.meta().exists());
+
+        SnapshotListTaskResult res = listSnapshots(grid(0));
+
+        int foundSnpsCnt = 0;
+        boolean victimNodeFound = false;
+
+        for (int i = 0; i < res.nodesIds().length; i++) {
+            UUID nid = res.nodesIds()[i];
+
+            foundSnpsCnt += res.nodesSnapshots()[i].snapshots().size();
+
+            if (nid.equals(grid(testGridIdx).localNode().id())) {
+                victimNodeFound = true;
+
+                assertTrue(res.nodesSnapshots()[i].snapshots().isEmpty());
+            }
+        }
+
+        assertTrue(victimNodeFound);
+        assertEquals(grids - 1, foundSnpsCnt);
+    }
+
+    /** */
+    @Test
+    public void testMissingIncrementalMeta() throws Exception {
+        doTestWithWrongIncrementalMeta(false);
+    }
+
+    /** */
+    @Test
+    public void testCorruptedIncrementalMeta() throws Exception {
+        doTestWithWrongIncrementalMeta(true);
+    }
+
+    /**
+     * Tests snapshot list when incremental snapshot metadata cannot be read.
+     * The main snapshot should still be listed, but without incremental info on the affected node.
+     *
+     * @param corruptFile If {@code true}, corrupts metadata. Otherwise, deletes metadata.
+     */
+    private void doTestWithWrongIncrementalMeta(boolean corruptFile) throws Exception {
+        // Incremental snapshots do not support the only-primary mode or encryption.
+        assumeFalse(onlyPrimary || encryption);
+
+        int grids = 3;
+        int testGridIdx = 1;
+        int incsCnt = 3;
+
+        IgniteEx ig = startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        snp(ig).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary).get(getTestTimeout());
+
+        for (int i = 0; i < incsCnt; ++i) {
+            try (IgniteDataStreamer<Integer, Integer> ds = grid(0).dataStreamer(DEFAULT_CACHE_NAME)) {
+                for (int kv = (i + 1) * CACHE_KEYS_RANGE; kv < (i + 1) * CACHE_KEYS_RANGE * 2; ++kv)
+                    ds.addData(kv, kv);
+            }
+
+            snp(ig).createSnapshot(SNAPSHOT_NAME, null, true, onlyPrimary).get(getTestTimeout());
+        }
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+        SnapshotFileTree.IncrementalSnapshotFileTree incFt = testSnpFt.incrementalSnapshotFileTree(1);
+        File incMeta = incFt.meta();
+
+        assertTrue(incMeta.exists());
+
+        if (corruptFile) {
+            try (RandomAccessFile raf = new RandomAccessFile(incMeta, "rw")) {
+                raf.write(UUID.randomUUID().toString().getBytes());
+            }
+        }
+        else
+            assertTrue(incMeta.delete() && !incMeta.exists());
+
+        SnapshotListTaskResult res = listSnapshots(grid(0));
+
+        for (int i = 0; i < res.nodesIds().length; i++) {
+            UUID nid = res.nodesIds()[i];
+
+            Map<String, SnapshotListJobResult.SnapshotInfo> snps = res.nodesSnapshots()[i].snapshots();
+
+            assertEquals(1, snps.size());
+
+            SnapshotListJobResult.SnapshotInfo info = snps.get(SNAPSHOT_NAME);
+
+            assertNotNull(info);
+            assertNotNull(info.incrementals());
+
+            if (nid.equals(grid(testGridIdx).localNode().id()))
+                assertEquals(incsCnt - 1, info.incrementals().number().intValue());
+            else
+                assertEquals(incsCnt, info.incrementals().number().intValue());
+        }
+    }
+
+    /**
+     * Test snapshot list operation when a node can't read some snapshot part due to insufficient permissions.
+     * I.e. a test node is able to read snapshot meta but can't read some of the snapshot's data.
+     */
+    @Test
+    public void testDeniedPermissions() throws Exception {
+        assumeTrue(posixPermissions);
+        // We rely on the snapshot sizes here. Better to avoid empty data nodes not to become flaky.
+        assumeFalse(onlyPrimary);
+
+        int grids = 3;
+        int testGridIdx = 1;
+
+        // Permissions to restore.
+        Map<Path, Set<PosixFilePermission>> oldPerms = new ConcurrentHashMap<>();
+        // The 'change permissions' flag.
+        AtomicBoolean changePermissions = new AtomicBoolean(true);
+
+        // Deny reading on a couple of incremental snapshot metadata files on one node.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft,
+                            boolean failIfCantRead) {
+                            if (changePermissions.get() && ctx.localNode().order() == testGridIdx + 1) {
+                                File victimDir = sft.nodeStorage();
+
+                                assertTrue(victimDir.exists());
+                                assertTrue(victimDir.isDirectory());
+
+                                // Ensure that blocked snapshot part has some size. We use sizes to compare later.
+                                try {
+                                    assertTrue(SnapshotListTask.calculateDirectorySize(victimDir) > 0L);
+                                }
+                                catch (IOException e) {
+                                    throw new IllegalStateException(e);
+                                }
+
+                                Path victimDirPath = victimDir.toPath();
+
+                                try {
+                                    Set<PosixFilePermission> perms = Files.getPosixFilePermissions(victimDirPath);
+
+                                    assertFalse(perms.isEmpty());
+
+                                    // Deny reading.
+                                    Files.setPosixFilePermissions(victimDirPath, PosixFilePermissions.fromString("---------"));
+
+                                    // Save actual permissions to restore.
+                                    oldPerms.put(victimDirPath, perms);
+                                }
+                                catch (Exception e) {
+                                    throw new IgniteException("Unable to set the test posix permissions.", e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, true);
+
+        UUID testNodeId = grid(testGridIdx).localNode().id();
+
+        // Snapshot size with restricted permissions.
+        long testSize0 = 0L;
+        // Snapshot size with normal permissions.
+        long testSize1 = 0L;
+
+        SnapshotListTaskResult snpLstOpRes;
+
+        try {
+            // First run.
+            snpLstOpRes = listSnapshots(grid(0));
+
+            assertFalse(oldPerms.isEmpty());
+
+            for (int i = 0; i < snpLstOpRes.nodesIds().length; i++) {
+                UUID nodeId = snpLstOpRes.nodesIds()[i];
+
+                // Store size of the partly read snapshot.
+                if (nodeId.equals(testNodeId)) {
+                    Map<String, SnapshotListJobResult.SnapshotInfo> snps = snpLstOpRes.nodesSnapshots()[i].snapshots();
+
+                    assertEquals(1, snps.size());
+
+                    testSize0 = snps.get(SNAPSHOT_NAME).size();
+                }
+            }
+
+            // Ensure that we've found and read snapshot on the test node.
+            assertTrue(testSize0 > 0L);
+        }
+        finally {
+            // Restore the permissions in any case.
+            oldPerms.forEach((path, perms) -> {
+                try {
+                    Files.setPosixFilePermissions(path, perms);
+                }
+                catch (IOException e) {
+                    throw new IllegalStateException(e);
+                }
+            });
+        }
+
+        // Relaunch the operation.
+        changePermissions.set(false);
+
+        snpLstOpRes = listSnapshots(grid(0));
+
+        for (int i = 0; i < snpLstOpRes.nodesIds().length; i++) {
+            UUID nodeId = snpLstOpRes.nodesIds()[i];
+
+            // Store size of the partly read snapshot.
+            if (nodeId.equals(testNodeId)) {
+                Map<String, SnapshotListJobResult.SnapshotInfo> snps = snpLstOpRes.nodesSnapshots()[i].snapshots();
+
+                assertEquals(1, snps.size());
+
+                testSize1 = snps.get(SNAPSHOT_NAME).size();
+            }
+        }
+
+        // Ensure that the calculated anew size is bigger than in the previous run.

Review Comment:
   typo



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324269047


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {

Review Comment:
   Done. Now is in `GridFileUtils`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324377652


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));

Review Comment:
   Normally, we throw NodeStoppingException when the local node is stopping. Unfortunately, it extends IgniteCheckedException, which I dislike, so sometimes it needs to be wrapped. I unified a bit, see `checkNodeIsStopping()`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324382616


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+            // Read local snapshots.
+            List<T2<SnapshotFileTree, Long>> locSnps = findLocalSnapshots(arg.src());
+
+            if (locSnps.isEmpty())
+                return new SnapshotListJobResult(Collections.emptyMap());
+
+            // Main sizes of the local snapshots (per snapshot name).
+            Map<String, Long> sizes = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional incremental parts descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> incs = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional external storages descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> extStors = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+
+            // Snapshot sizes, incremental parts and external storages futures.
+            List<Future<?>> futs = new ArrayList<>(locSnps.size() * 3);
+            // Names of snapshots with read failures.
+            Set<String> failedSnps = ConcurrentHashMap.newKeySet(locSnps.size() / 2);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            // Descriptions of the local snapshots: main sizes, external storages and incremental parts.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                SnapshotFileTree sft = snpPair.get1();
+                String snpName = sft.name();
+
+                // Future for the snapshot main size. It is the largest part of the job, thus it goes first.
+                Future<?> fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));

Review Comment:
   The same answer.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324529656


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/CoreFeatureRegistry.java:
##########
@@ -93,4 +93,7 @@
 public class CoreFeatureRegistry {
     /** */
     public static final IgniteFeature ROLLING_UPGRADE_FEATURE = new IgniteCoreFeature(0);
+
+    /** */
+    public static final IgniteFeature SNAPSHOT_LIST_FEATURE = new IgniteCoreFeature(1);

Review Comment:
   Ignore. The testing features are for the tests. Mostly, not related to this ticket and this feture. We have a dedicated RU test. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6213021513

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsSingleMessage.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6252823101

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsSingleMessage.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6256135107

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsSingleMessage.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6250131652

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsSingleMessage.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324462784


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+            // Read local snapshots.
+            List<T2<SnapshotFileTree, Long>> locSnps = findLocalSnapshots(arg.src());
+
+            if (locSnps.isEmpty())
+                return new SnapshotListJobResult(Collections.emptyMap());
+
+            // Main sizes of the local snapshots (per snapshot name).
+            Map<String, Long> sizes = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional incremental parts descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> incs = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional external storages descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> extStors = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+
+            // Snapshot sizes, incremental parts and external storages futures.
+            List<Future<?>> futs = new ArrayList<>(locSnps.size() * 3);
+            // Names of snapshots with read failures.
+            Set<String> failedSnps = ConcurrentHashMap.newKeySet(locSnps.size() / 2);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            // Descriptions of the local snapshots: main sizes, external storages and incremental parts.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                SnapshotFileTree sft = snpPair.get1();
+                String snpName = sft.name();
+
+                // Future for the snapshot main size. It is the largest part of the job, thus it goes first.
+                Future<?> fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        sizes.put(snpName, calculateDirectorySize(sft.root()));
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to calculate snapshot's size, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional incremental parts.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo incDesc = incrementals(sft);
+
+                        if (incDesc != null)
+                            incs.put(snpName, incDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's incremental parts, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional external storages.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo extDesc = externalStorages(sft);
+
+                        if (extDesc != null)
+                            extStors.put(snpName, extDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's external storages, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+            }
+
+            // Wait for the futures.
+            for (Future<?> fut : futs) {
+                try {
+                    fut.get();
+                }
+                catch (ExecutionException e) {
+                    throw new IgniteException("Failed to read local node's snapshots.", e);
+                }
+                catch (InterruptedException e) {
+                    Thread.currentThread().interrupt();
+
+                    throw new IgniteException("Interrupted while reading local snapshots.", e);
+                }
+            }
+
+            // Result snapshot descriptions.
+            Map<String, SnapshotListJobResult.SnapshotInfo> resMap = new ConcurrentHashMap<>(locSnps.size() - failedSnps.size(), 1.0f);
+
+            // Reduce results.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                String snpName = snpPair.get1().name();
+
+                if (failedSnps.contains(snpName))
+                    continue;
+
+                Long size = sizes.get(snpName);
+
+                // The size must be calculated unless the snapshot has failed.
+                assert size != null;
+
+                SnapshotListJobResult.SnapshotInfo snpDesc = new SnapshotListJobResult.SnapshotInfo(
+                    size,
+                    snpPair.get2(),
+                    extStors.get(snpName),
+                    incs.get(snpName)
+                );
+
+                resMap.put(snpName, snpDesc);
+            }
+
+            return new SnapshotListJobResult(resMap);
+        }
+
+        /** */
+        private @Nullable SnapshotListJobResult.SnapshotInfo externalStorages(SnapshotFileTree sft) {
+            int extStoragesCnt = 0;
+            long extStoragesSize = 0;
+
+            for (File extraStorage : sft.allStorages().toList()) {
+                if (sft.nodeStorage().equals(extraStorage))
+                    continue;
+
+                try {
+                    extStoragesSize += calculateDirectorySize(extraStorage);
+                }
+                catch (Exception e) {
+                    log.warning("Failed to calculate snapshot's external storage size, storage ignored [extraStorage='" +
+                        extraStorage + "']", e);
+
+                    continue;
+                }
+
+                extStoragesCnt++;
+            }
+
+            return extStoragesCnt == 0 ? null : new SnapshotListJobResult.SnapshotInfo(extStoragesCnt, extStoragesSize);
+        }
+
+        /** @return Snapshot file tree and creation time from the snapshot metadata. */
+        private List<T2<SnapshotFileTree, Long>> findLocalSnapshots(@Nullable String snpPath) {
+            // The tree is used only to extract the snapshots root directory. Such snapshot name is actually unallowed.
+            File[] dirsToParse = new SnapshotFileTree(ignite.context(), "snp", snpPath).root().getParentFile().listFiles();

Review Comment:
   There is exact comment above for this:
   `// The tree is used only to extract the snapshots root directory. The snapshot name used is not essential.`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4323613112


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommand.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Comparator;
+import java.util.TreeMap;
+import java.util.function.Consumer;
+import java.util.stream.IntStream;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.util.typedef.internal.U;
+
+/** Snapshot list command. */
+public class SnapshotListCommand extends AbstractSnapshotCommand<SnapshotListCommandArg, SnapshotListTaskResult> {
+    /** */
+    public static final String HEADER = "Snapshots found on the following nodes:";
+
+    /** */
+    public static final String DESC = "Lists all snapshots on all online server nodes with their sizes";
+
+    /** */
+    public static final String NO_SNAPSHOTS = "No snapshots found.";
+
+    /** */
+    private static final String PATTERN_FORMAT = "yyyy-MM-dd HH:mm:ss Z";
+
+    /** */
+    private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern(PATTERN_FORMAT).withZone(ZoneId.systemDefault());
+
+    /** {@inheritDoc} */
+    @Override public String description() {
+        return DESC;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListCommandArg> argClass() {
+        return SnapshotListCommandArg.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public Class<SnapshotListTask> taskClass() {
+        return SnapshotListTask.class;
+    }
+
+    /** {@inheritDoc} */
+    @Override public void printResult(SnapshotListCommandArg arg, SnapshotListTaskResult res, Consumer<String> printer) {
+        printer.accept(HEADER);
+
+        // Sorting the results by consistent id for better reading.
+        Integer[] nodesOrder = IntStream.range(0, res.nodesIds().length).boxed()
+            .sorted(Comparator.comparing(idx -> res.consistentIds()[idx]))
+            .toArray(Integer[]::new);
+
+        for (int nodeIdx : nodesOrder) {
+            // Skip line before node.
+            printer.accept("");

Review Comment:
   With the additional line several snapshots are likes like this:
   
   ```
   --------------------------------------------------------------------------------
   Snapshots were found on the following nodes:
   
   	Node 'gridCommandHandlerTest0' [uuid=8ccee03c-b7e9-4c6d-84e0-a0c55de00000]:
   		Snapshot 'testSnapshot0': totalSize=671.0 KB (687131b), created='2026-10-20 12:40:23 +0300' (epoch=1792489223301)
   		Snapshot 'testSnapshot1': totalSize=671.0 KB (687131b), created='2026-10-20 12:40:23 +0300' (epoch=1792489223527)
   		Snapshot 'testSnapshot2': totalSize=671.0 KB (687131b), created='2026-10-20 12:40:23 +0300' (epoch=1792489223561)
   
   	Node 'gridCommandHandlerTest1' [uuid=ef54ce0e-1df1-4d53-9431-c22bfb200001]:
   		Snapshot 'testSnapshot0': totalSize=671.0 KB (687131b), created='2026-10-20 12:40:23 +0300' (epoch=1792489223301)
   		Snapshot 'testSnapshot1': totalSize=671.0 KB (687131b), created='2026-10-20 12:40:23 +0300' (epoch=1792489223527)
   		Snapshot 'testSnapshot2': totalSize=671.0 KB (687131b), created='2026-10-20 12:40:23 +0300' (epoch=1792489223561)
   
   	Node 'gridCommandHandlerTest2' [uuid=50bfec94-7461-4094-801c-e05931d00002]:
   		Snapshot 'testSnapshot0': totalSize=671.0 KB (687131b), created='2026-10-20 12:40:23 +0300' (epoch=1792489223301)
   		Snapshot 'testSnapshot1': totalSize=671.0 KB (687131b), created='2026-10-20 12:40:23 +0300' (epoch=1792489223527)
   		Snapshot 'testSnapshot2': totalSize=671.0 KB (687131b), created='2026-10-20 12:40:23 +0300' (epoch=1792489223561)
   
   Command [SNAPSHOT] finished with code: 0
   ```
   
   This empty line splits the nodes. WDYT?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324511959


##########
modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListTask.java:
##########
@@ -0,0 +1,452 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.management.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.compute.ComputeJobResult;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.IncrementalSnapshotMetadata;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListJobResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotListTaskResult;
+import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotMetadata;
+import org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry;
+import org.apache.ignite.internal.processors.task.GridInternal;
+import org.apache.ignite.internal.thread.pool.IgniteThreadPoolExecutor;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.T2;
+import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.visor.VisorJob;
+import org.apache.ignite.internal.visor.VisorMultiNodeTask;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.resources.LoggerResource;
+import org.jetbrains.annotations.Nullable;
+
+import static org.apache.ignite.internal.util.lang.ClusterNodeFunc.nodeIds;
+
+/** */
+@GridInternal
+public class SnapshotListTask extends VisorMultiNodeTask<SnapshotListCommandArg, SnapshotListTaskResult, SnapshotListJobResult> {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** */
+    public static final String FEATURE_NOT_ACTIVE_ERR_MSG = "Won't search for local snapshots. " +
+        "The snapshot list feature isn't activated yet.";
+
+    /** {@inheritDoc} */
+    @Override protected VisorJob<SnapshotListCommandArg, SnapshotListJobResult> job(SnapshotListCommandArg arg) {
+        return new SnapshotListJob(arg, debug);
+    }
+
+    /** {@inheritDoc} */
+    @Override protected Collection<UUID> jobNodes(VisorTaskArgument<SnapshotListCommandArg> arg) {
+        if (!ignite.context().rollingUpgrade().features().isActive(CoreFeatureRegistry.SNAPSHOT_LIST_FEATURE))
+            throw new IgniteException(FEATURE_NOT_ACTIVE_ERR_MSG);
+
+        /** Allows {@link #map0(List, VisorTaskArgument)} to use the entire grid. */
+        return nodeIds(ignite.cluster().forServers().nodes());
+    }
+
+    /** {@inheritDoc} */
+    @Override protected SnapshotListTaskResult reduce0(List<ComputeJobResult> nodesJobsResults) throws IgniteException {
+        String[] cstIds = new String[nodesJobsResults.size()];
+        UUID[] nodesIds = new UUID[nodesJobsResults.size()];
+        SnapshotListJobResult[] nodesResults = new SnapshotListJobResult[nodesJobsResults.size()];
+
+        for (int i = 0; i < nodesJobsResults.size(); i++) {
+            ComputeJobResult nodeJobRes = nodesJobsResults.get(i);
+
+            if (nodeJobRes.getException() != null) {
+                throw new IgniteException("Failed to execute snapshot list job on node [uuid=" + nodeJobRes.getNode().id() + ']',
+                    nodeJobRes.getException());
+            }
+
+            assert nodeJobRes.getData() != null;
+
+            cstIds[i] = nodeConsistentId(nodeJobRes.getNode());
+            nodesIds[i] = nodeJobRes.getNode().id();
+            nodesResults[i] = nodeJobRes.getData();
+        }
+
+        return new SnapshotListTaskResult(cstIds, nodesIds, nodesResults);
+    }
+
+    /** */
+    private String nodeConsistentId(ClusterNode n) {
+        UUID nodeId = n.id();
+
+        n = ignite.context().discovery().node(nodeId);
+
+        if (n == null)
+            n = ignite.context().discovery().historicalNode(nodeId);
+
+        return n == null ? "" : n.consistentId().toString();
+    }
+
+    /**
+     * Walk through a directory. Doesn't lock it or its content. Tries to find files and summarize their size.
+     * Tolerates and skips concurrent modification errors.
+     */
+    public static long calculateDirectorySize(File path) throws IOException {
+        AtomicLong size = new AtomicLong(0);
+        AtomicBoolean entered = new AtomicBoolean();
+
+        Files.walkFileTree(path.toPath(), new SimpleFileVisitor<>() {
+            @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
+                entered.compareAndSet(false, true);
+
+                // Use attrs instead of Files.size() for efficiency.
+                if (attrs.isRegularFile())
+                    size.addAndGet(attrs.size());
+
+                return FileVisitResult.CONTINUE;
+            }
+
+            @Override public FileVisitResult visitFileFailed(Path file, IOException err) throws IOException {
+                if (!entered.get() && file.toFile().equals(path)) {
+                    // Cannot even start snapshot size calculation - can't enter snapshot directory.
+                    throw err;
+                }
+
+                return FileVisitResult.CONTINUE;
+            }
+        });
+
+        return size.get();
+    }
+
+    /** */
+    private static class SnapshotListJob extends SnapshotJob<SnapshotListCommandArg, SnapshotListJobResult> {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** */
+        @LoggerResource
+        private IgniteLogger log;
+
+        /**
+         * @param arg Snapshot list task argument.
+         * @param debug Flag indicating whether debug information should be printed into node log.
+         */
+        protected SnapshotListJob(SnapshotListCommandArg arg, boolean debug) {
+            super(arg, debug);
+        }
+
+        /** {@inheritDoc} */
+        @Override protected SnapshotListJobResult run(SnapshotListCommandArg arg) {
+            assert !ignite.localNode().isClient();
+
+            if (ignite.context().isStopping())
+                throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+            // Read local snapshots.
+            List<T2<SnapshotFileTree, Long>> locSnps = findLocalSnapshots(arg.src());
+
+            if (locSnps.isEmpty())
+                return new SnapshotListJobResult(Collections.emptyMap());
+
+            // Main sizes of the local snapshots (per snapshot name).
+            Map<String, Long> sizes = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional incremental parts descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> incs = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+            // Optional external storages descriptions (per snapshot name).
+            Map<String, SnapshotListJobResult.SnapshotInfo> extStors = new ConcurrentHashMap<>(locSnps.size(), 1.0f);
+
+            // Snapshot sizes, incremental parts and external storages futures.
+            List<Future<?>> futs = new ArrayList<>(locSnps.size() * 3);
+            // Names of snapshots with read failures.
+            Set<String> failedSnps = ConcurrentHashMap.newKeySet(locSnps.size() / 2);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            // Descriptions of the local snapshots: main sizes, external storages and incremental parts.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                SnapshotFileTree sft = snpPair.get1();
+                String snpName = sft.name();
+
+                // Future for the snapshot main size. It is the largest part of the job, thus it goes first.
+                Future<?> fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        sizes.put(snpName, calculateDirectorySize(sft.root()));
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to calculate snapshot's size, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional incremental parts.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo incDesc = incrementals(sft);
+
+                        if (incDesc != null)
+                            incs.put(snpName, incDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's incremental parts, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+
+                // Future for optional external storages.
+                fut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    if (failedSnps.contains(snpName))
+                        return;
+
+                    try {
+                        SnapshotListJobResult.SnapshotInfo extDesc = externalStorages(sft);
+
+                        if (extDesc != null)
+                            extStors.put(snpName, extDesc);
+                    }
+                    catch (Exception e) {
+                        failedSnps.add(snpName);
+
+                        log.warning("Failed to read snapshot's external storages, snapshot ignored [snpName='" + snpName + "']", e);
+                    }
+                });
+
+                futs.add(fut);
+            }
+
+            // Wait for the futures.
+            for (Future<?> fut : futs) {
+                try {
+                    fut.get();
+                }
+                catch (ExecutionException e) {
+                    throw new IgniteException("Failed to read local node's snapshots.", e);
+                }
+                catch (InterruptedException e) {
+                    Thread.currentThread().interrupt();
+
+                    throw new IgniteException("Interrupted while reading local snapshots.", e);
+                }
+            }
+
+            // Result snapshot descriptions.
+            Map<String, SnapshotListJobResult.SnapshotInfo> resMap = new ConcurrentHashMap<>(locSnps.size() - failedSnps.size(), 1.0f);
+
+            // Reduce results.
+            for (T2<SnapshotFileTree, Long> snpPair : locSnps) {
+                String snpName = snpPair.get1().name();
+
+                if (failedSnps.contains(snpName))
+                    continue;
+
+                Long size = sizes.get(snpName);
+
+                // The size must be calculated unless the snapshot has failed.
+                assert size != null;
+
+                SnapshotListJobResult.SnapshotInfo snpDesc = new SnapshotListJobResult.SnapshotInfo(
+                    size,
+                    snpPair.get2(),
+                    extStors.get(snpName),
+                    incs.get(snpName)
+                );
+
+                resMap.put(snpName, snpDesc);
+            }
+
+            return new SnapshotListJobResult(resMap);
+        }
+
+        /** */
+        private @Nullable SnapshotListJobResult.SnapshotInfo externalStorages(SnapshotFileTree sft) {
+            int extStoragesCnt = 0;
+            long extStoragesSize = 0;
+
+            for (File extraStorage : sft.allStorages().toList()) {
+                if (sft.nodeStorage().equals(extraStorage))
+                    continue;
+
+                try {
+                    extStoragesSize += calculateDirectorySize(extraStorage);
+                }
+                catch (Exception e) {
+                    log.warning("Failed to calculate snapshot's external storage size, storage ignored [extraStorage='" +
+                        extraStorage + "']", e);
+
+                    continue;
+                }
+
+                extStoragesCnt++;
+            }
+
+            return extStoragesCnt == 0 ? null : new SnapshotListJobResult.SnapshotInfo(extStoragesCnt, extStoragesSize);
+        }
+
+        /** @return Snapshot file tree and creation time from the snapshot metadata. */
+        private List<T2<SnapshotFileTree, Long>> findLocalSnapshots(@Nullable String snpPath) {
+            // The tree is used only to extract the snapshots root directory. Such snapshot name is actually unallowed.
+            File[] dirsToParse = new SnapshotFileTree(ignite.context(), "snp", snpPath).root().getParentFile().listFiles();
+
+            if (F.isEmpty(dirsToParse))
+                return Collections.emptyList();
+
+            List<Future<T2<SnapshotFileTree, Long>>> futs = new ArrayList<>(dirsToParse.length);
+
+            IgniteThreadPoolExecutor exec = ignite.context().pools().getSnapshotExecutorService();
+
+            for (File snpDir : dirsToParse) {
+                Future<T2<SnapshotFileTree, Long>> snpDirFut = exec.submit(() -> {
+                    if (ignite.context().isStopping())
+                        throw new IgniteException("Won't search for local snapshots.", new NodeStoppingException("Node is stopping."));
+
+                    String snpName = snpDir.getName();
+
+                    // Snapshot tree being used as a path, to read the metas only.
+                    SnapshotFileTree sft = new SnapshotFileTree(ignite.context(), snpName, snpPath);
+
+                    List<SnapshotMetadata> metas = ignite.context().cache().context().snapshotMgr().readSnapshotMetadatas(sft, false);
+
+                    if (metas.isEmpty())
+                        return null;
+
+                    // Get the last-created time snapshot metadata.
+                    SnapshotMetadata snpMeta = metas.stream().max(Comparator.comparingLong(SnapshotMetadata::snapshotTime)).get();
+
+                    // Real, meta-based snapshot file tree. Can belong to other cluster, other consistent id.

Review Comment:
   Fixed the comment slightly. First things first: we find the snapshot directory and search for snapshots there. But we can find a snapshot taken from an old cluster configuration — old consistent IDs, node names, and so on. We have to read exactly this snapshot with its correct directory names. So we take its real snapshot file tree, which is built on the metadata we actually read.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324518858


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.util.UUID;
+import org.apache.ignite.internal.Order;
+import org.apache.ignite.internal.dto.IgniteDataTransferObject;
+
+/** Accumulated result of the snapshot list task. */
+public final class SnapshotListTaskResult extends IgniteDataTransferObject {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** Nodes consistent ids. */
+    @Order(0)
+    String[] cstIds;

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6304440338

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsSingleMessage.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6096670138

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsSingleMessage.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6096698734

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsSingleMessage.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324670062


##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListTest.java:
##########
@@ -0,0 +1,936 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.io.Serializable;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Stream;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.NodeStoppingException;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.lang.IgniteFuture;
+import org.apache.ignite.plugin.AbstractTestPluginProvider;
+import org.apache.ignite.plugin.PluginConfiguration;
+import org.apache.ignite.plugin.PluginContext;
+import org.apache.ignite.plugin.PluginProvider;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.junit.runners.Parameterized;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.configuration.IgniteConfiguration.DFLT_SNAPSHOT_THREAD_POOL_SIZE;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.cartesianProduct;
+import static org.apache.ignite.testframework.GridTestUtils.runAsync;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Cluster-wide snapshot list procedure tests. */
+public class IgniteClusterSnapshotListTest extends AbstractSnapshotSelfTest {
+    /** Number of cache keys to pre-create at node startup. */
+    private static final int CACHE_KEYS_RANGE = 15;
+
+    /** Number of partitions of the test cache. */
+    private static final int CACHE_PARTITIONS_COUNT = 4;
+
+    /** Flag showing that the filesystem supports POSIX permissions. */
+    private static boolean posixPermissions;
+
+    /** Size of the snapshot utility thread pool. */
+    @Parameterized.Parameter(2)
+    public int snpThrdPoolSz;
+
+    /** */
+    private PluginProvider<PluginConfiguration> pluginProvider;
+
+    /** Flag to spread the test cache data over the external storages. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@link #extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Parameters. */
+    @Parameterized.Parameters(name = "encryption={0}, onlyPrimary={1}, snpThrdPoolSz={2}")
+    public static Collection<Object[]> params() {
+        return cartesianProduct(
+            encryptionParameters(), // Encryption.
+            F.asList(false, true), // Only primary.
+            F.asList(DFLT_SNAPSHOT_THREAD_POOL_SIZE, 1) // Snapshots thread pool size.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName)
+            .setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        if (pluginProvider != null)
+            cfg.setPluginProviders(pluginProvider);
+
+        if (extStorages) {
+            // External storage paths must be identical on all the nodes (a cache has the single storage paths
+            // setting). Thus, a shared work directory is used, like in GridCommandHandlerListSnapshotTest.
+            cfg.setWorkDirectory(U.defaultWorkDirectory());
+
+            cfg.getDataStorageConfiguration().setExtraStoragePaths(
+                U.defaultWorkDirectory() + File.separator,
+                U.defaultWorkDirectory() + File.separator + "extStorage"
+            );
+
+            extStoragePaths = cfg.getDataStorageConfiguration().getExtraStoragePaths();
+
+            cfg.getDataStorageConfiguration().setExtraSnapshotPaths("", "extStorage");
+        }
+
+        cfg.setSnapshotThreadPoolSize(snpThrdPoolSz);
+
+        return cfg;
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        super.cleanPersistenceDir();
+
+        // Also cleans separated snapshot working directories and custom snapshot paths.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTestsStarted() throws Exception {
+        super.beforeTestsStarted();
+
+        File workDir = new File(U.defaultWorkDirectory());
+
+        assertTrue(workDir.exists());
+
+        try {
+            Files.getPosixFilePermissions(workDir.toPath());
+
+            posixPermissions = true;
+        }
+        catch (UnsupportedOperationException ignored) {
+            // No-op.
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected <K, V> CacheConfiguration<K, V> txCacheConfig(CacheConfiguration<K, V> ccfg) {
+        // Speeds up the tests.
+        ccfg = super.txCacheConfig(ccfg)
+            .setAffinity(new RendezvousAffinityFunction(false, CACHE_PARTITIONS_COUNT))
+            .setBackups(1);
+
+        if (extStorages) {
+            assert !F.isEmpty(extStoragePaths);
+
+            ccfg.setStoragePaths(extStoragePaths);
+        }
+
+        return ccfg;
+    }
+
+    /** */
+    @Test
+    public void testConcurrentCreation() throws Exception {
+        // The test uses the thread blocking and conditional waiting. Won't proceed with 1 thread.
+        assumeTrue(snpThrdPoolSz > 1);
+
+        int grids = 3;
+        int testNodeIdx = 1;
+        int testNodeOrder = testNodeIdx + 1;
+
+        CountDownLatch beginSnpCreation = new CountDownLatch(grids);
+        CountDownLatch proceedSnpCreation = new CountDownLatch(1);
+
+        // Delays snapshot creation after its metadata is written.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public <M extends Serializable> void storeSnapshotMeta(M meta, File smf) {
+                            super.storeSnapshotMeta(meta, smf);
+
+                            beginSnpCreation.countDown();
+
+                            if (((IgniteEx)ctx.grid()).localNode().order() == testNodeOrder) {
+                                try {
+                                    assertTrue(proceedSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        IgniteFuture<Void> createSnpFut = snp(grid(0)).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary);
+
+        assertTrue(beginSnpCreation.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        SnapshotListTaskResult lstOpRes = listSnapshots(grid(0));
+
+        int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+        assertEquals(grids, snpsCnt);
+
+        SnapshotFileTree testSft = new SnapshotFileTree(grid(testNodeIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(SnapshotListTask.calculateDirectorySize(testSft.root()) > 0L);
+
+        proceedSnpCreation.countDown();
+
+        createSnpFut.get(getTestTimeout());
+    }
+
+    /** */
+    @Test
+    public void testCompletelyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(true);
+    }
+
+    /** */
+    @Test
+    public void testPartlyDeletedAfterMetaRead() throws Exception {
+        doTestDeletionAfterMetaRead(false);
+    }
+
+    /** */
+    private void doTestDeletionAfterMetaRead(boolean completeDeletion) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        CountDownLatch metaReadBeginLatch = new CountDownLatch(grids);
+        CountDownLatch metaReadProceedLatch = new CountDownLatch(1);
+
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft, boolean failIfCantRead) {
+                            metaReadBeginLatch.countDown();
+
+                            if (cctx.localNode().order() == testGridIdx + 1) {
+                                try {
+                                    assertTrue(metaReadProceedLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+                                }
+                                catch (InterruptedException e) {
+                                    throw new IllegalStateException(e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        long testSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+        assertTrue(testSnpSz > 0);
+
+        IgniteInternalFuture<SnapshotListTaskResult> lstOpFut = runAsync(() -> listSnapshots(grid(0)));
+
+        assertTrue(metaReadBeginLatch.await(getTestTimeout(), TimeUnit.MILLISECONDS));
+
+        if (completeDeletion)
+            assertTrue(testSnpFt.root().exists() && U.delete(testSnpFt.root()) && !testSnpFt.root().exists());
+        else
+            assertTrue(testSnpFt.nodeStorage().exists() && U.delete(testSnpFt.nodeStorage()) && !testSnpFt.nodeStorage().exists());
+
+        metaReadProceedLatch.countDown();
+
+        SnapshotListTaskResult lstOpRes = lstOpFut.get();
+
+        if (completeDeletion) {
+            int snpsCnt = Stream.of(lstOpRes.nodesSnapshots()).mapToInt(jr -> jr.snapshots().size()).sum();
+
+            assertEquals(grids - 1, snpsCnt);
+        }
+        else {
+            int snpsCnt = 0;
+            boolean victimNodeFound = false;
+
+            for (int i = 0; i < lstOpRes.nodesIds().length; i++) {
+                UUID nid = lstOpRes.nodesIds()[i];
+
+                Map<String, SnapshotListJobResult.SnapshotInfo> nodeSnps = lstOpRes.nodesSnapshots()[i].snapshots();
+
+                snpsCnt += nodeSnps.size();
+
+                if (nid.equals(grid(testGridIdx).localNode().id())) {
+                    victimNodeFound = true;
+
+                    assertEquals(1, nodeSnps.size());
+
+                    long curSnpSz = SnapshotListTask.calculateDirectorySize(testSnpFt.root());
+
+                    assertTrue(curSnpSz < testSnpSz);
+                }
+            }
+
+            assertTrue(victimNodeFound);
+            assertEquals(grids, snpsCnt);
+        }
+    }
+
+    /** */
+    @Test
+    public void testMissingSnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(true);
+    }
+
+    /** */
+    @Test
+    public void testEmptySnapshotPath() throws Exception {
+        doTestWrongSnapshotPath(false);
+    }
+
+    /** */
+    private void doTestWrongSnapshotPath(boolean missing) throws Exception {
+        // Doesn't matter here, speeds up the tests.
+        assumeFalse(snpThrdPoolSz < 2);
+
+        doTestSnapshotPath(null);
+
+        File path = new File(U.defaultWorkDirectory(), "not_snapshots");
+
+        if (!missing)
+            assertTrue(new File(path, SNAPSHOT_NAME).mkdirs());
+
+        SnapshotListJobResult[] lstOpRes = listSnapshots(grid(0), path.getAbsolutePath()).nodesSnapshots();
+
+        int cnt = Stream.of(lstOpRes).mapToInt(nodeRes -> nodeRes.snapshots().size()).sum();
+
+        assertEquals(0, cnt);
+    }
+
+    /** */
+    @Test
+    public void testDefaultSnapshotPath() throws Exception {
+        doTestSnapshotPath(null);
+    }
+
+    /** */
+    @Test
+    @Ignore("https://issues.apache.org/jira/browse/IGNITE-29126")
+    public void testRelativeSnapshotPath() throws Exception {
+        doTestSnapshotPath("ex_snapshots");
+    }
+
+    /** */
+    @Test
+    public void testAbsoluteSnapshotPath() throws Exception {
+        doTestSnapshotPath(new File(U.defaultWorkDirectory(), "ex_snapshots").getAbsolutePath());
+    }
+
+    /** */
+    private void doTestSnapshotPath(@Nullable String path) throws Exception {
+        int grids = 3;
+
+        startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        snp(grid(0)).createSnapshot(SNAPSHOT_NAME, path, false, onlyPrimary).get(getTestTimeout());
+
+        SnapshotListJobResult[] lstOpRes = listSnapshots(grid(0), path).nodesSnapshots();
+
+        int cnt = Stream.of(lstOpRes).mapToInt(nodeRes -> nodeRes.snapshots().size()).sum();
+
+        assertEquals(grids, cnt);
+    }
+
+    /** */
+    @Test
+    public void testMissingMeta() throws Exception {
+        doTestWithWrongMeta(false);
+    }
+
+    /** */
+    @Test
+    public void testCorruptedMeta() throws Exception {
+        doTestWithWrongMeta(true);
+    }
+
+    /**
+     * Tests snapshot lists when one snapshot metadata cannot be read.
+     *
+     * @param corruptFile If {@code true}, corrupts metadata. Otherwise, deletes metadata.
+     */
+    private void doTestWithWrongMeta(boolean corruptFile) throws Exception {
+        int grids = 3;
+        int testGridIdx = 1;
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, false);
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+
+        assertTrue(testSnpFt.meta().exists());
+
+        if (corruptFile) {
+            try (RandomAccessFile raf = new RandomAccessFile(testSnpFt.meta(), "rw")) {
+                raf.write(UUID.randomUUID().toString().getBytes());
+            }
+        }
+        else
+            assertTrue(testSnpFt.meta().delete() && !testSnpFt.meta().exists());
+
+        SnapshotListTaskResult res = listSnapshots(grid(0));
+
+        int foundSnpsCnt = 0;
+        boolean victimNodeFound = false;
+
+        for (int i = 0; i < res.nodesIds().length; i++) {
+            UUID nid = res.nodesIds()[i];
+
+            foundSnpsCnt += res.nodesSnapshots()[i].snapshots().size();
+
+            if (nid.equals(grid(testGridIdx).localNode().id())) {
+                victimNodeFound = true;
+
+                assertTrue(res.nodesSnapshots()[i].snapshots().isEmpty());
+            }
+        }
+
+        assertTrue(victimNodeFound);
+        assertEquals(grids - 1, foundSnpsCnt);
+    }
+
+    /** */
+    @Test
+    public void testMissingIncrementalMeta() throws Exception {
+        doTestWithWrongIncrementalMeta(false);
+    }
+
+    /** */
+    @Test
+    public void testCorruptedIncrementalMeta() throws Exception {
+        doTestWithWrongIncrementalMeta(true);
+    }
+
+    /**
+     * Tests snapshot list when incremental snapshot metadata cannot be read.
+     * The main snapshot should still be listed, but without incremental info on the affected node.
+     *
+     * @param corruptFile If {@code true}, corrupts metadata. Otherwise, deletes metadata.
+     */
+    private void doTestWithWrongIncrementalMeta(boolean corruptFile) throws Exception {
+        // Incremental snapshots do not support the only-primary mode or encryption.
+        assumeFalse(onlyPrimary || encryption);
+
+        int grids = 3;
+        int testGridIdx = 1;
+        int incsCnt = 3;
+
+        IgniteEx ig = startGridsWithCache(grids, txCacheConfig(defaultCacheConfiguration()), CACHE_KEYS_RANGE);
+
+        snp(ig).createSnapshot(SNAPSHOT_NAME, null, false, onlyPrimary).get(getTestTimeout());
+
+        for (int i = 0; i < incsCnt; ++i) {
+            try (IgniteDataStreamer<Integer, Integer> ds = grid(0).dataStreamer(DEFAULT_CACHE_NAME)) {
+                for (int kv = (i + 1) * CACHE_KEYS_RANGE; kv < (i + 1) * CACHE_KEYS_RANGE * 2; ++kv)
+                    ds.addData(kv, kv);
+            }
+
+            snp(ig).createSnapshot(SNAPSHOT_NAME, null, true, onlyPrimary).get(getTestTimeout());
+        }
+
+        SnapshotFileTree testSnpFt = new SnapshotFileTree(grid(testGridIdx).context(), SNAPSHOT_NAME, null);
+        SnapshotFileTree.IncrementalSnapshotFileTree incFt = testSnpFt.incrementalSnapshotFileTree(1);
+        File incMeta = incFt.meta();
+
+        assertTrue(incMeta.exists());
+
+        if (corruptFile) {
+            try (RandomAccessFile raf = new RandomAccessFile(incMeta, "rw")) {
+                raf.write(UUID.randomUUID().toString().getBytes());
+            }
+        }
+        else
+            assertTrue(incMeta.delete() && !incMeta.exists());
+
+        SnapshotListTaskResult res = listSnapshots(grid(0));
+
+        for (int i = 0; i < res.nodesIds().length; i++) {
+            UUID nid = res.nodesIds()[i];
+
+            Map<String, SnapshotListJobResult.SnapshotInfo> snps = res.nodesSnapshots()[i].snapshots();
+
+            assertEquals(1, snps.size());
+
+            SnapshotListJobResult.SnapshotInfo info = snps.get(SNAPSHOT_NAME);
+
+            assertNotNull(info);
+            assertNotNull(info.incrementals());
+
+            if (nid.equals(grid(testGridIdx).localNode().id()))
+                assertEquals(incsCnt - 1, info.incrementals().number().intValue());
+            else
+                assertEquals(incsCnt, info.incrementals().number().intValue());
+        }
+    }
+
+    /**
+     * Test snapshot list operation when a node can't read some snapshot part due to insufficient permissions.
+     * I.e. a test node is able to read snapshot meta but can't read some of the snapshot's data.
+     */
+    @Test
+    public void testDeniedPermissions() throws Exception {
+        assumeTrue(posixPermissions);
+        // We rely on the snapshot sizes here. Better to avoid empty data nodes not to become flaky.
+        assumeFalse(onlyPrimary);
+
+        int grids = 3;
+        int testGridIdx = 1;
+
+        // Permissions to restore.
+        Map<Path, Set<PosixFilePermission>> oldPerms = new ConcurrentHashMap<>();
+        // The 'change permissions' flag.
+        AtomicBoolean changePermissions = new AtomicBoolean(true);
+
+        // Deny reading on a couple of incremental snapshot metadata files on one node.
+        pluginProvider = new AbstractTestPluginProvider() {
+            @Override public String name() {
+                return "TestSnpMgrProvider";
+            }
+
+            @Override public <T> T createComponent(PluginContext ctx, Class<T> cls) {
+                if (IgniteSnapshotManager.class.isAssignableFrom(cls)) {
+                    return (T)new IgniteSnapshotManager(((IgniteEx)ctx.grid()).context()) {
+                        @Override public List<SnapshotMetadata> readSnapshotMetadatas(SnapshotFileTree sft,
+                            boolean failIfCantRead) {
+                            if (changePermissions.get() && ctx.localNode().order() == testGridIdx + 1) {
+                                File victimDir = sft.nodeStorage();
+
+                                assertTrue(victimDir.exists());
+                                assertTrue(victimDir.isDirectory());
+
+                                // Ensure that blocked snapshot part has some size. We use sizes to compare later.
+                                try {
+                                    assertTrue(SnapshotListTask.calculateDirectorySize(victimDir) > 0L);
+                                }
+                                catch (IOException e) {
+                                    throw new IllegalStateException(e);
+                                }
+
+                                Path victimDirPath = victimDir.toPath();
+
+                                try {
+                                    Set<PosixFilePermission> perms = Files.getPosixFilePermissions(victimDirPath);
+
+                                    assertFalse(perms.isEmpty());
+
+                                    // Deny reading.
+                                    Files.setPosixFilePermissions(victimDirPath, PosixFilePermissions.fromString("---------"));
+
+                                    // Save actual permissions to restore.
+                                    oldPerms.put(victimDirPath, perms);
+                                }
+                                catch (Exception e) {
+                                    throw new IgniteException("Unable to set the test posix permissions.", e);
+                                }
+                            }
+
+                            return super.readSnapshotMetadatas(sft, failIfCantRead);
+                        }
+                    };
+                }
+
+                return super.createComponent(ctx, cls);
+            }
+        };
+
+        startGridsWithSnapshot(grids, CACHE_KEYS_RANGE, true, true);
+
+        UUID testNodeId = grid(testGridIdx).localNode().id();
+
+        // Snapshot size with restricted permissions.
+        long testSize0 = 0L;
+        // Snapshot size with normal permissions.
+        long testSize1 = 0L;
+
+        SnapshotListTaskResult snpLstOpRes;
+
+        try {
+            // First run.
+            snpLstOpRes = listSnapshots(grid(0));
+
+            assertFalse(oldPerms.isEmpty());
+
+            for (int i = 0; i < snpLstOpRes.nodesIds().length; i++) {
+                UUID nodeId = snpLstOpRes.nodesIds()[i];
+
+                // Store size of the partly read snapshot.
+                if (nodeId.equals(testNodeId)) {
+                    Map<String, SnapshotListJobResult.SnapshotInfo> snps = snpLstOpRes.nodesSnapshots()[i].snapshots();
+
+                    assertEquals(1, snps.size());
+
+                    testSize0 = snps.get(SNAPSHOT_NAME).size();
+                }
+            }
+
+            // Ensure that we've found and read snapshot on the test node.
+            assertTrue(testSize0 > 0L);
+        }
+        finally {
+            // Restore the permissions in any case.
+            oldPerms.forEach((path, perms) -> {
+                try {
+                    Files.setPosixFilePermissions(path, perms);
+                }
+                catch (IOException e) {
+                    throw new IllegalStateException(e);
+                }
+            });
+        }
+
+        // Relaunch the operation.
+        changePermissions.set(false);
+
+        snpLstOpRes = listSnapshots(grid(0));
+
+        for (int i = 0; i < snpLstOpRes.nodesIds().length; i++) {
+            UUID nodeId = snpLstOpRes.nodesIds()[i];
+
+            // Store size of the partly read snapshot.
+            if (nodeId.equals(testNodeId)) {
+                Map<String, SnapshotListJobResult.SnapshotInfo> snps = snpLstOpRes.nodesSnapshots()[i].snapshots();
+
+                assertEquals(1, snps.size());
+
+                testSize1 = snps.get(SNAPSHOT_NAME).size();
+            }
+        }
+
+        // Ensure that the calculated anew size is bigger than in the previous run.

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4324678461


##########
modules/core/src/test/resources/org.apache.ignite.util/GridCommandHandlerClusterByClassTest_help.output:
##########
@@ -269,6 +269,12 @@ This utility can do the following commands:
   Get the status of the current snapshot operation:
     control.(sh|bat) --snapshot status
 
+  Lists all snapshots on all online server nodes with their sizes:

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4375865848


##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/IgniteClusterSnapshotListRollingUpgradeTest.java:
##########
@@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.io.File;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.UUID;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.cache.CacheAtomicityMode;
+import org.apache.ignite.cache.CacheMode;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.DataRegionConfiguration;
+import org.apache.ignite.configuration.DataStorageConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommandArg;
+import org.apache.ignite.internal.management.snapshot.SnapshotListTask;
+import org.apache.ignite.internal.processors.rollingupgrade.AbstractRollingUpgradeTest;
+import org.apache.ignite.internal.util.distributed.SingleNodeMessage;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.internal.visor.VisorTaskArgument;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.junit.Test;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.internal.TestRecordingCommunicationSpi.spi;
+import static org.apache.ignite.internal.util.distributed.DistributedProcess.DistributedProcessType.RU_PREPARE_VERSION_FINALIZATION;
+import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+
+/** */
+public class IgniteClusterSnapshotListRollingUpgradeTest extends AbstractRollingUpgradeTest {
+    /** */
+    private static final int ALL_GRIDS = 4;
+
+    /** */
+    private static final int CLIENTS = 1;
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        super.afterTest();
+
+        // Clean everything, including the separated working directories.
+        try (DirectoryStream<Path> files = newDirectoryStream(Paths.get(U.defaultWorkDirectory()))) {
+            for (Path path : files)
+                U.delete(path);
+        }
+    }
+
+    /** {@inheritDoc} */
+    @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName, String ver) throws Exception {
+        IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName, ver);
+
+        DataRegionConfiguration dataRegionCfg = new DataRegionConfiguration()
+            .setPersistenceEnabled(true)
+            .setMaxSize(DataStorageConfiguration.DFLT_DATA_REGION_INITIAL_SIZE);
+
+        cfg.setDataStorageConfiguration(new DataStorageConfiguration().setDefaultDataRegionConfiguration(dataRegionCfg));
+
+        cfg.setWorkDirectory(new File(U.defaultWorkDirectory(), igniteInstanceName).getAbsolutePath());
+
+        return cfg;
+    }
+
+    /** */
+    @Test
+    public void testParallelRollingUpgradeInProgress() throws Exception {
+        for (int i = 0; i < ALL_GRIDS; i++)
+            startGrid(i, "2.18.0", i >= ALL_GRIDS - CLIENTS);
+
+        grid(0).cluster().active(true);
+
+        int testNodeIx = ALL_GRIDS - CLIENTS - 1;
+
+        createCacheAndSnapshot(testNodeIx);
+
+        ru(grid(testNodeIx)).enableVersionUpgrade();
+
+        for (int i = 0; i < ALL_GRIDS; i++) {
+            assertTrue(ru(grid(i)).isVersionUpgradeEnabled());
+
+            upgradeNodeVersion(i, "2.18.0", MASTER_VER_REF);
+        }
+
+        spi(grid(testNodeIx)).blockMessages((node, msg) -> msg instanceof SingleNodeMessage<?> snm &&
+            snm.type() == RU_PREPARE_VERSION_FINALIZATION.ordinal());
+
+        IgniteInternalFuture<?> finalizeFut = GridTestUtils.runAsync(() -> ru(testNodeIx).finalizeClusterVersion());
+
+        assertTrue(spi(grid(testNodeIx)).waitForBlocked(1, getTestTimeout()));
+
+        ensureSnapshotListCommandFails();
+
+        spi(grid(testNodeIx)).stopBlock();
+
+        assertFalse(spi(grid(testNodeIx)).hasBlockedMessages());
+
+        finalizeFut.get(getTestTimeout());
+
+        for (int i = 0; i < ALL_GRIDS; i++) {
+            int i0 = i;
+
+            assertTrue(waitForCondition(() -> !ru(grid(i0)).isVersionUpgradeEnabled(), getTestTimeout()));
+        }
+
+        ensureSnapshotListSucceeds();
+    }
+
+    /** */
+    @Test
+    public void testSnapshotListFeature() throws Exception {
+        for (int i = 0; i < ALL_GRIDS; i++)
+            startGrid(i, "2.18.0", i >= ALL_GRIDS - CLIENTS);
+
+        grid(0).cluster().active(true);
+
+        createCacheAndSnapshot(1);
+
+        ensureSnapshotListCommandFails();
+
+        ru(grid(0)).enableVersionUpgrade();
+
+        for (int i = 0; i < ALL_GRIDS; i++) {
+            assertTrue(ru(grid(i)).isVersionUpgradeEnabled());
+
+            upgradeNodeVersion(i, "2.18.0", MASTER_VER_REF);
+
+            ensureSnapshotListCommandFails();
+        }
+
+        ru(grid(1)).finalizeClusterVersion();
+
+        for (int i = 0; i < ALL_GRIDS; i++) {
+            int i0 = i;
+
+            assertTrue(waitForCondition(() -> !ru(grid(i0)).isVersionUpgradeEnabled(), getTestTimeout()));
+        }
+
+        ensureSnapshotListSucceeds();
+    }
+
+    /** */
+    private void ensureSnapshotListSucceeds() throws Exception {
+        for (int i = 0; i < ALL_GRIDS; i++) {
+            for (int k = 0; k < ALL_GRIDS; k++) {
+                int i0 = i;
+                int k0 = k;

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4375719348


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java:
##########
@@ -0,0 +1,140 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.snapshot;
+
+import java.util.Map;
+import org.apache.ignite.internal.Order;
+import org.apache.ignite.internal.dto.IgniteDataTransferObject;
+import org.jetbrains.annotations.Nullable;
+
+/** Per-node result of the snapshot list task. Contains information about the snapshots found on the node. */
+public final class SnapshotListJobResult extends IgniteDataTransferObject {
+    /** Serial version uid. */
+    private static final long serialVersionUID = 0L;
+
+    /** Local node's snapshot descriptions. */
+    @Order(0)
+    Map<String, SnapshotInfo> snapshots;
+
+    /** Default constructor for serialization purposes. */
+    public SnapshotListJobResult() {
+        // No-op.
+    }
+
+    /** */
+    public SnapshotListJobResult(Map<String, SnapshotInfo> snapshots) {
+        this.snapshots = snapshots;
+    }
+
+    /** @return Snapshot descriptions by snapshot name. */
+    public Map<String, SnapshotInfo> snapshots() {
+        return snapshots;
+    }
+
+    /** Holds information about a snapshot or its parts: size, creation time, number of incremental parts or external storages. */
+    public static class SnapshotInfo extends IgniteDataTransferObject {
+        /** Serial version uid. */
+        private static final long serialVersionUID = 0L;
+
+        /** Total size of a snapshot. Or size of its external storages or incremental parts. */
+        @Order(0)
+        long size;
+
+        /** Creation date of the snapshot or of its incremental parts. */
+        @Order(1)
+        @Nullable Long date;

Review Comment:
   > unboxing on constructor and boxing here, why do we need it ?
   
   The constructors create:
   
   - The main description. A snapshot has a creation date, so we do not expect it to be `@Nullable` in the constructor.
   - Description of incremental snapshot parts. Incremental snapshots can be created later. So, they have their own date, and we always pass it to the constructor. No need to be `@Nullable` and boxed in the constructor.
   - Description of snapshot external storages. There is no date in this constructor because we do not print dates for external storages. The storages come with the main snapshot data. So, the date variable is `@Nullable` for external storages. Looks fair.
   
   In contrast, `cnt` is used only for external storages and incremental snapshots. So it is `@Nullable` for the main snapshot data but is not expected to be @Nullable in the constructors for external storages and incremental snapshots.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6365603826

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13666:
URL: https://github.com/apache/ignite/pull/13666#issuecomment-6059204813

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/management/snapshot/SnapshotListCommandArg.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListJobResult.java`
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotListTaskResult.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4310738537


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/CoreFeatureRegistry.java:
##########
@@ -93,4 +93,7 @@
 public class CoreFeatureRegistry {
     /** */
     public static final IgniteFeature ROLLING_UPGRADE_FEATURE = new IgniteCoreFeature(0);
+
+    /** */
+    public static final IgniteFeature SNAPSHOT_LIST_FEATURE = new IgniteCoreFeature(1);

Review Comment:
   Feature ID 1 is already assigned to `VER_2_19_0_ID_1_FEATURE` in the existing 2.19.0–2.19.3 release registries (`modules/core/src/test/java/org/apache/ignite/internal/processors/rollingupgrade/feature/TestIgniteReleaseFeatures_2_19_3.java:26`). Feature IDs are the compatibility identifiers, so reusing it makes a 2.19.3 node appear to support `SNAPSHOT_LIST_FEATURE` even though it has no list task, allowing the command during rolling upgrade. Allocate a new unused monotonically increasing ID and update the release feature definitions/table accordingly.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29056 Introduce snapshot lists command to control.sh [ignite]

Posted by "Vladsz83 (via GitHub)" <gi...@apache.org>.
Vladsz83 commented on code in PR #13666:
URL: https://github.com/apache/ignite/pull/13666#discussion_r4323501189


##########
modules/control-utility/src/test/java/org/apache/ignite/util/GridCommandHandlerListSnapshotTest.java:
##########
@@ -0,0 +1,348 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.util;
+
+import java.io.File;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Collection;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteDataStreamer;
+import org.apache.ignite.cluster.ClusterNode;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.DataStorageConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.management.snapshot.SnapshotListCommand;
+import org.apache.ignite.internal.processors.cache.persistence.filename.SnapshotFileTree;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.G;
+import org.apache.ignite.internal.util.typedef.internal.U;
+import org.apache.ignite.lang.IgnitePredicate;
+import org.apache.ignite.testframework.GridTestUtils;
+import org.jetbrains.annotations.Nullable;
+import org.junit.Test;
+import org.junit.runners.Parameterized.Parameter;
+import org.junit.runners.Parameterized.Parameters;
+
+import static java.nio.file.Files.newDirectoryStream;
+import static org.apache.ignite.cluster.ClusterState.ACTIVE;
+import static org.apache.ignite.internal.commandline.CommandHandler.EXIT_CODE_OK;
+import static org.apache.ignite.internal.processors.cache.persistence.snapshot.AbstractSnapshotSelfTest.snp;
+import static org.junit.Assume.assumeFalse;
+import static org.junit.Assume.assumeTrue;
+
+/** Test for the command '--snapshot list'. */
+public class GridCommandHandlerListSnapshotTest extends GridCommandHandlerAbstractTest {
+    /** External storage path. */
+    private static final String EXT_STORAGE_PATH = "extStorage";
+
+    /** Flag to use {@link DataStorageConfiguration#setExtraSnapshotPaths(String...)}. */
+    private boolean extStorages;
+
+    /** Resolved external storages paths. {@code null} if {@code extStorages} is {@code false}. */
+    private @Nullable String[] extStoragePaths;
+
+    /** Consistent id postfix. */
+    private @Nullable String cstIdPostfix = "";
+
+    /** Flag setting the usage of a custom snapshot path. */
+    @Parameter(1)
+    public boolean customPath;
+
+    /** Flag setting the usage of dedicated, own node working directories. */
+    @Parameter(2)
+    public boolean separatedWorkDir;
+
+    /** Flag to add external server node after the snapshot creation. */
+    @Parameter(3)
+    public boolean addExtraSrvr;
+
+    /** Number of incremental snapshots to add to the main test snapshots. */
+    @Parameter(4)
+    public int incCnt;
+
+    /** */
+    @Parameters(name = "cmdHnd={0}, customPath={1}, ownWorkDir={2}, addExtraSrvr={3}, incCnt={4}")
+    public static Collection<?> parameters() {
+        return GridTestUtils.cartesianProduct(
+            commandHandlers(),
+            F.asList(false, true), // Custom snapshot path.
+            F.asList(false, true), // Separated (own) work directories.
+            F.asList(false, true), // Add a server node after the snapshot creation.
+            F.asList(0, 2) // Number of incremental snapshots.
+        );
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void afterTest() throws Exception {
+        super.afterTest();
+
+        stopAllGrids();
+
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void beforeTest() throws Exception {
+        super.beforeTest();
+
+        /** Handy if test running is interrupted and {@link #afterTest()} isn't invoked. */
+        cleanPersistenceDir();
+    }
+
+    /** {@inheritDoc} */
+    @Override protected void cleanPersistenceDir() throws Exception {
+        // Also cleans separated snapshot working directories and custom snapshot paths.

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org