You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by "nvharikrishna (via GitHub)" <gi...@apache.org> on 2026/01/25 11:32:03 UTC

[PR] CASSSIDECAR-226 Adding endpoint for verifying files post data copy during live migration [cassandra-sidecar]

nvharikrishna opened a new pull request, #309:
URL: https://github.com/apache/cassandra-sidecar/pull/309

   CASSSIDECAR-226 Adding an endpoint for verifying files between source and destination post data copy.
   
   Here are the endpoint details:
   
   **Sample files verification task submission request:**
   ```bash
   curl -X POST http://dest-host.example.com:9043/api/v1/live-migration/files-verification-tasks \
     -H "Content-Type: application/json" \
     -d '{
       "maxConcurrency": 10,
       "digestAlgorithm": "MD5"
     }'
   ```
   
   It supports XXHash32 algorithm too and `seed` as additional input in the payload.
   
   **Sample response:**
   
   ```bash
   {
     "taskId": "b8e4f3d2-5c6b-5d9e-0f2g-3b4c5d6e7f8g",
     "statusUrl": "/api/v1/live-migration/files-verification-tasks/b8e4f3d2-5c6b-5d9e-0f2g-3b4c5d6e7f8g"
   }
   ```
   
   **Fetching files verification task status**
   ```bash
   curl -X GET http://dest-host.example.com:9043/api/v1/live-migration/files-verification-tasks/b8e4f3d2-5c6b-5d9e-0f2g-3b4c5d6e7f8g
   ```
   
   **Sample response:**
   ```bash
   {
     "id": "b8e4f3d2-5c6b-5d9e-0f2g-3b4c5d6e7f8g ",
     "digestAlgorithm": "md5",
     "seed": null,
     "state": "COMPLETED",
     "source": "localhost1",
     "port": 9043,
     "filesNotFoundAtSource": 0,
     "filesNotFoundAtDestination": 0,
     "metadataMatched": 379,
     "metadataMismatches": 0,
     "digestMismatches": 0,
     "digestVerificationFailures": 0,
     "filesMatched": 323
   }
   ```
   
   Also made additional changes to ensure that either data copy task or file verification task can be executed at any point of time.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSSIDECAR-226 Adding endpoint for verifying files post data copy during live migration [cassandra-sidecar]

Posted by "frankgh (via GitHub)" <gi...@apache.org>.
frankgh commented on code in PR #309:
URL: https://github.com/apache/cassandra-sidecar/pull/309#discussion_r3213431247


##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationTaskManager.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+
+import com.google.common.annotations.VisibleForTesting;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import org.apache.cassandra.sidecar.cluster.InstancesMetadata;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Centralized singleton manager for all live migration tasks across Cassandra instances.
+ * Enforces the constraint that only one {@link LiveMigrationTask} (of any type) can be active per instance at a time,
+ * preventing concurrent data copy and files verification operations that could lead to resource conflicts or data integrity issues.
+ * This singleton is shared across {@link DataCopyTaskManager} and {@link FilesVerificationTaskManager} to ensure
+ * proper coordination and mutual exclusion of tasks per instance.
+ */
+@Singleton
+public class LiveMigrationTaskManager
+{
+    @VisibleForTesting
+    final ConcurrentHashMap<Integer, LiveMigrationTask<?>> currentTasks = new ConcurrentHashMap<>();
+
+    private final InstancesMetadata instancesMetadata;
+
+    @Inject
+    public LiveMigrationTaskManager(InstancesMetadata instancesMetadata)
+    {
+        this.instancesMetadata = instancesMetadata;
+    }
+
+    /**
+     * Attempts to submit a new task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time.
+     *
+     * @param instanceId the instance ID
+     * @param newTask    the task to submit
+     * @return true if the task was accepted, false if another task is already in progress
+     */
+    public boolean submitTask(int instanceId, LiveMigrationTask<?> newTask)
+    {
+        return currentTasks.compute(instanceId, (integer, taskInMap) -> {
+            if (taskInMap == null)
+            {
+                return newTask;
+            }
+
+            if (!taskInMap.isCompleted())
+            {
+                // Reject new task if existing task is still in progress
+                return taskInMap;
+            }
+            else
+            {
+                // Accept new task if existing task has completed
+                return newTask;
+            }
+        }) == newTask;
+    }
+
+    /**
+     * Returns all live migration tasks for given currentHost.
+     * This includes both active and completed tasks that haven't been replaced.
+     *
+     * @param currentHost the host where sidecar is running
+     * @return list containing at most one task (empty if no task has ever been submitted for this host)
+     */
+    @SuppressWarnings("ConstantValue")
+    public List<LiveMigrationTask<?>> getAllTasks(@NotNull String currentHost)
+    {
+        InstanceMetadata localInstance = instancesMetadata.instanceFromHost(currentHost);
+        if (localInstance == null)
+        {
+            throw new IllegalStateException("No instance found for host: " + currentHost);
+        }
+
+        LiveMigrationTask<?> task = currentTasks.get(localInstance.id());
+        return task == null ? Collections.emptyList() : Collections.singletonList(task);
+    }
+
+    /**
+     * Returns the live migration task with the specified task ID.
+     *
+     * @param taskId      ID of the task to retrieve
+     * @param currentHost the host where sidecar is running
+     * @return the LiveMigrationTask matching the given taskId
+     * @throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException if no task found with the given ID
+     */
+    public LiveMigrationTask<?> getTask(@NotNull String taskId,
+                                        @NotNull String currentHost) throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException
+    {
+        return getLiveMigrationTask(taskId, currentHost);
+    }
+
+    /**
+     * Cancels the live migration task with the specified task ID.
+     *
+     * @param taskId      ID of the task to cancel
+     * @param currentHost the host where sidecar is running
+     * @return the cancelled LiveMigrationTask
+     * @throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException if no task found with the given ID
+     */
+    public LiveMigrationTask<?> cancelTask(@NotNull String taskId,
+                                           @NotNull String currentHost) throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException
+    {
+        LiveMigrationTask<?> taskInProgress = getLiveMigrationTask(taskId, currentHost);
+
+        // Cancelling the task
+        taskInProgress.cancel();
+
+        return taskInProgress;
+    }
+
+    @SuppressWarnings("ConstantValue")
+    private LiveMigrationTask<?> getLiveMigrationTask(@NotNull String taskId, @NotNull String currentHost)
+    {
+        InstanceMetadata localInstance = instancesMetadata.instanceFromHost(currentHost);
+        if (localInstance == null)
+        {
+            throw new IllegalStateException("No instance found for host: " + currentHost);
+        }

Review Comment:
   I still see the null check here, can you check?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSSIDECAR-226 Adding endpoint for verifying files post data copy during live migration [cassandra-sidecar]

Posted by "yifan-c (via GitHub)" <gi...@apache.org>.
yifan-c commented on code in PR #309:
URL: https://github.com/apache/cassandra-sidecar/pull/309#discussion_r2808177605


##########
client-common/src/main/java/org/apache/cassandra/sidecar/common/request/LiveMigrationFileDigestRequest.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.common.request;
+
+import java.util.Objects;
+
+import io.netty.handler.codec.http.HttpMethod;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+
+/**
+ * Represents a request to retrieve file digest for validation during live migration.
+ * Supports configurable digest algorithms and optional seed values for verification.
+ */
+public class LiveMigrationFileDigestRequest extends JsonRequest<DigestResponse>
+{
+    public static final String DIGEST_ALGORITHM_PARAM = "digestAlgorithm";
+    public static final String SEED_PARAM = "seed";
+
+    /**
+     * Private constructor for internal use by the factory method
+     *
+     * @param fullRequestURI the complete URI with query parameters
+     */
+    private LiveMigrationFileDigestRequest(String fullRequestURI)
+    {
+        super(fullRequestURI);
+    }
+
+    /**
+     * Creates a live migration file digest request with validation
+     *
+     * @param requestURI the base URI of the request
+     * @param digestAlgorithm the digest algorithm to use (e.g., "MD5", "SHA-256")
+     * @param seed       optional seed value for digest verification, may be null
+     * @return a new LiveMigrationFileDigestRequest instance
+     * @throws NullPointerException     if requestURI or digestAlgorithm is null
+     * @throws IllegalArgumentException if digestAlgorithm is empty
+     */
+    public static LiveMigrationFileDigestRequest create(String requestURI, String digestAlgorithm, Integer seed)
+    {
+        Objects.requireNonNull(requestURI, "requestURI cannot be null");
+        Objects.requireNonNull(digestAlgorithm, "digestAlgorithm cannot be null");
+        if (digestAlgorithm.isEmpty())
+        {
+            throw new IllegalArgumentException("digestAlgorithm cannot be empty");
+        }
+
+        String fullURI = seed != null
+                         ? String.format("%s?%s=%s&%s=%d", requestURI, DIGEST_ALGORITHM_PARAM, digestAlgorithm, SEED_PARAM, seed)

Review Comment:
   One learning from the RestoreJob work is that the custom seed does not provide benefit for data integrity validation, but only adds code complexity. I would just drop the support of custom seed support to simplify the implementation, and use the fixed seed 0, which also makes the client-server communication simpler. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSSIDECAR-226 Adding endpoint for verifying files post data copy during live migration [cassandra-sidecar]

Posted by "frankgh (via GitHub)" <gi...@apache.org>.
frankgh commented on code in PR #309:
URL: https://github.com/apache/cassandra-sidecar/pull/309#discussion_r3170657610


##########
client-common/src/main/java/org/apache/cassandra/sidecar/common/response/DigestResponse.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.common.response;
+
+import java.util.Objects;
+
+import com.fasterxml.jackson.annotation.JsonCreator;
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import org.apache.cassandra.sidecar.common.request.data.Digest;
+import org.apache.cassandra.sidecar.common.request.data.MD5Digest;
+import org.apache.cassandra.sidecar.common.request.data.XXHash32Digest;
+
+/**
+ * Response object containing a cryptographic digest value for file verification purposes.
+ */
+public class DigestResponse
+{
+    @JsonProperty("digest")
+    public final String digest;
+    @JsonProperty("digestAlgorithm")
+    public final String digestAlgorithm;
+
+    @JsonCreator
+    public DigestResponse(@JsonProperty("digest") String digest,
+                          @JsonProperty("digestAlgorithm") String digestAlgorithm)
+    {
+        Objects.requireNonNull(digest, "digest is required");
+        Objects.requireNonNull(digestAlgorithm, "digestAlgorithm is required");
+        this.digest = digest;
+        this.digestAlgorithm = digestAlgorithm;
+    }

Review Comment:
   very small NIT, no need to change it
   ```suggestion
           this.digest = Objects.requireNonNull(digest, "digest is required");
           this.digestAlgorithm = Objects.requireNonNull(digestAlgorithm, "digestAlgorithm is required");
   ```



##########
client-common/src/main/java/org/apache/cassandra/sidecar/common/response/LiveMigrationFilesVerificationResponse.java:
##########
@@ -0,0 +1,186 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.common.response;
+
+import java.util.Objects;
+
+import com.fasterxml.jackson.annotation.JsonCreator;
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+/**
+ * Response object for live migration file verification operations, containing statistics about
+ * files not found at source/target and digest mismatches during verification.
+ */
+@JsonIgnoreProperties(ignoreUnknown = true)
+public class LiveMigrationFilesVerificationResponse
+{
+    private final String id;
+    private final String state;
+    private final String source;
+    private final int port;
+    private final int filesNotFoundAtSource;
+    private final int filesNotFoundAtDestination;
+    private final int metadataMatched;
+    private final int metadataMismatches;
+    private final int digestMismatches;
+    private final int digestVerificationFailures;
+    private final int filesMatched;
+    private final String digestAlgorithm;
+
+    @JsonCreator
+    public LiveMigrationFilesVerificationResponse(@JsonProperty("id") String id,
+                                                  @JsonProperty("digestAlgorithm") String digestAlgorithm,
+                                                  @JsonProperty("state") String state,
+                                                  @JsonProperty("source") String source,
+                                                  @JsonProperty("port") int port,
+                                                  @JsonProperty("filesNotFoundAtSource") int filesNotFoundAtSource,
+                                                  @JsonProperty("filesNotFoundAtDestination") int filesNotFoundAtDestination,
+                                                  @JsonProperty("metadataMatched") int metadataMatched,
+                                                  @JsonProperty("metadataMismatches") int metadataMismatches,
+                                                  @JsonProperty("digestMismatches") int digestMismatches,
+                                                  @JsonProperty("digestVerificationFailures") int digestVerificationFailures,
+                                                  @JsonProperty("filesMatched") int filesMatched)
+    {
+        Objects.requireNonNull(id, "id of files verification task must be specified");
+        Objects.requireNonNull(state, "state of files verification task must be specified");
+        this.id = id;
+        this.digestAlgorithm = digestAlgorithm;
+        this.state = state;

Review Comment:
   Small NIT, feel free to ignore:
   ```suggestion
           this.id = Objects.requireNonNull(id, "id of files verification task must be specified");
           this.digestAlgorithm = digestAlgorithm;
           this.state = Objects.requireNonNull(state, "state of files verification task must be specified");
   ```



##########
CHANGES.txt:
##########
@@ -1,5 +1,6 @@
 0.3.0
 -----
+ * Adding endpoint for verifying files post data copy during live migration (CASSSIDECAR-226)

Review Comment:
   should move under 0.4.0



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCreateFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.json.DecodeException;
+import io.vertx.core.json.Json;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationTaskCreationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationInvalidRequestException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.NoSuchCassandraInstanceException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.ACCEPTED;
+import static io.netty.handler.codec.http.HttpResponseStatus.BAD_REQUEST;
+import static io.netty.handler.codec.http.HttpResponseStatus.CONFLICT;
+import static io.netty.handler.codec.http.HttpResponseStatus.SERVICE_UNAVAILABLE;
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.common.ApiEndpointsV1.LIVE_MIGRATION_FILES_VERIFICATION_TASKS_ROUTE;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * HTTP handler for creating file digest verification tasks during live migration.
+ * Manages concurrent verification tasks per instance and orchestrates the verification process.
+ */
+public class LiveMigrationCreateFilesVerificationTaskHandler extends AbstractHandler<LiveMigrationFilesVerificationRequest> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationCreateFilesVerificationTaskHandler.class);
+
+    private final FilesVerificationTaskManager filesVerificationTaskManager;
+    private final LiveMigrationMap liveMigrationMap;
+
+    /**
+     * Constructs a handler with the provided {@code metadataFetcher}
+     *
+     * @param metadataFetcher the interface to retrieve instance metadata
+     * @param executorPools   the executor pools for blocking executions
+     * @param validator       a validator instance to validate Cassandra-specific input
+     */
+    @Inject
+    protected LiveMigrationCreateFilesVerificationTaskHandler(InstanceMetadataFetcher metadataFetcher,
+                                                              ExecutorPools executorPools,
+                                                              CassandraInputValidator validator,
+                                                              LiveMigrationMap liveMigrationMap,
+                                                              FilesVerificationTaskManager filesVerificationTaskManager)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.liveMigrationMap = liveMigrationMap;
+        this.filesVerificationTaskManager = filesVerificationTaskManager;
+    }
+
+    @Override
+    protected LiveMigrationFilesVerificationRequest extractParamsOrThrow(RoutingContext context)
+    {
+        try
+        {
+            return Json.decodeValue(context.body().buffer(), LiveMigrationFilesVerificationRequest.class);
+        }
+        catch (DecodeException decodeException)
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST,
+                                    "Failed to parse request body, please ensure that the request is valid.",
+                                    decodeException);
+        }
+        catch (IllegalArgumentException e)
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST, e.getMessage(), e);
+        }
+    }
+
+    @Override
+    protected void handleInternal(RoutingContext context,
+                                  HttpServerRequest httpRequest,
+                                  @NotNull String host,
+                                  SocketAddress remoteAddress,
+                                  LiveMigrationFilesVerificationRequest request)
+    {
+        LOGGER.debug("Received files verification request for host {} with maxConcurrency {}",
+                     host, request.maxConcurrency());
+        InstanceMetadata localInstanceMetadata;
+        try
+        {
+            localInstanceMetadata = metadataFetcher.instance(host);
+        }
+        catch (NoSuchCassandraInstanceException e)
+        {
+            LOGGER.error("Failed to fetch instance metadata for host={}", host);
+            context.fail(wrapHttpException(SERVICE_UNAVAILABLE, e));
+            return;
+        }

Review Comment:
   There's no need to handle these exceptions here. It is already explicitly handled in `org.apache.cassandra.sidecar.handlers.AbstractHandler#determineHttpException`
   ```suggestion
           InstanceMetadata localInstanceMetadata = metadataFetcher.instance(host);
   ```



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationGetFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.livemigration.LiveMigrationTask;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler for retrieving a specific live migration files verification task by task ID.
+ * Returns the task details if found, or a 404 error if the task does not exist on the specified host.
+ */
+public class LiveMigrationGetFilesVerificationTaskHandler extends AbstractHandler<String> implements AccessProtected

Review Comment:
   ```suggestion
   @Singleton
   public class LiveMigrationGetFilesVerificationTaskHandler extends AbstractHandler<String> implements AccessProtected
   ```



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCancelFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.livemigration.LiveMigrationTask;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler for canceling an active live migration files verification task.
+ * Accepts a task ID and cancels the corresponding verification task on the specified host.
+ */
+public class LiveMigrationCancelFilesVerificationTaskHandler extends AbstractHandler<String> implements AccessProtected

Review Comment:
   Annotate with Singleton
   ```suggestion
   @Singleton
   public class LiveMigrationCancelFilesVerificationTaskHandler extends AbstractHandler<String> implements AccessProtected
   ```



##########
client-common/src/test/java/org/apache/cassandra/sidecar/common/response/LiveMigrationFilesVerificationResponseTest.java:
##########
@@ -0,0 +1,322 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.common.response;
+
+import org.junit.jupiter.api.Test;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+class LiveMigrationFilesVerificationResponseTest
+{
+    private final ObjectMapper mapper = new ObjectMapper();
+
+    @Test
+    void testSerializationRoundTrip() throws Exception

Review Comment:
   we can simplify with a ParameterizedTest



##########
client-common/src/main/java/org/apache/cassandra/sidecar/common/response/LiveMigrationTaskCreationResponse.java:
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.common.response;
+
+import java.util.Objects;
+
+import com.fasterxml.jackson.annotation.JsonCreator;
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+/**
+ * Response object returned when a live migration task is created.
+ * Contains the task identifier and a URL to query the task status.
+ */
+public class LiveMigrationTaskCreationResponse
+{
+    private final String taskId;
+    private final String statusUrl;
+
+    @JsonCreator
+    public LiveMigrationTaskCreationResponse(@JsonProperty("taskId") String taskId,
+                                             @JsonProperty("statusUrl") String statusUrl)
+    {
+        Objects.requireNonNull(taskId, "taskId cannot be null");
+        Objects.requireNonNull(statusUrl, "statusUrl cannot be null");
+
+        this.taskId = taskId;
+        this.statusUrl = statusUrl;

Review Comment:
   Small NIT, feel free to ignore
   ```suggestion
           this.taskId = Objects.requireNonNull(taskId, "taskId cannot be null");
           this.statusUrl = Objects.requireNonNull(statusUrl, "statusUrl cannot be null");
   ```



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCreateFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.json.DecodeException;
+import io.vertx.core.json.Json;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationTaskCreationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationInvalidRequestException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.NoSuchCassandraInstanceException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.ACCEPTED;
+import static io.netty.handler.codec.http.HttpResponseStatus.BAD_REQUEST;
+import static io.netty.handler.codec.http.HttpResponseStatus.CONFLICT;
+import static io.netty.handler.codec.http.HttpResponseStatus.SERVICE_UNAVAILABLE;
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.common.ApiEndpointsV1.LIVE_MIGRATION_FILES_VERIFICATION_TASKS_ROUTE;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * HTTP handler for creating file digest verification tasks during live migration.
+ * Manages concurrent verification tasks per instance and orchestrates the verification process.
+ */
+public class LiveMigrationCreateFilesVerificationTaskHandler extends AbstractHandler<LiveMigrationFilesVerificationRequest> implements AccessProtected

Review Comment:
   ```suggestion
   @Singleton
   public class LiveMigrationCreateFilesVerificationTaskHandler extends AbstractHandler<LiveMigrationFilesVerificationRequest> implements AccessProtected
   ```



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCancelFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.livemigration.LiveMigrationTask;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler for canceling an active live migration files verification task.
+ * Accepts a task ID and cancels the corresponding verification task on the specified host.
+ */
+public class LiveMigrationCancelFilesVerificationTaskHandler extends AbstractHandler<String> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationCancelFilesVerificationTaskHandler.class);
+
+    private final FilesVerificationTaskManager taskManager;
+
+    /**
+     * Constructs a handler with the provided {@code metadataFetcher}
+     *
+     * @param metadataFetcher the interface to retrieve instance metadata
+     * @param executorPools   the executor pools for blocking executions
+     * @param validator       a validator instance to validate Cassandra-specific input
+     */
+    @Inject
+    public LiveMigrationCancelFilesVerificationTaskHandler(InstanceMetadataFetcher metadataFetcher,
+                                                           ExecutorPools executorPools,
+                                                           CassandraInputValidator validator,
+                                                           FilesVerificationTaskManager taskManager)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.taskManager = taskManager;
+    }
+
+    @Override
+    protected String extractParamsOrThrow(RoutingContext context)
+    {
+        return context.pathParam("taskId");

Review Comment:
   can we do some validation of this parameter here? I don't think it can be null, but can it be empty? Is there some parameter validation that can be done ?



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationConcurrencyLimitHandler.java:
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.Handler;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.concurrent.ConcurrencyLimiter;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler that enforces concurrency limits for live migration file operations.
+ * Returns HTTP 503 (SERVICE_UNAVAILABLE) when the maximum concurrent file requests limit is exceeded.
+ */
+@Singleton
+public class LiveMigrationConcurrencyLimitHandler implements Handler<RoutingContext>
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationConcurrencyLimitHandler.class);
+
+    private final ConcurrencyLimiter concurrencyLimiter;
+
+    @Inject
+    public LiveMigrationConcurrencyLimitHandler(SidecarConfiguration sidecarConfiguration)
+    {
+        this.concurrencyLimiter =
+        new ConcurrencyLimiter(() -> sidecarConfiguration.liveMigrationConfiguration().maxConcurrentFileRequests());
+    }
+
+    @Override
+    public void handle(RoutingContext rc)
+    {
+        if (!concurrencyLimiter.tryAcquire())
+        {
+            LOGGER.warn("Too many concurrent live migration file requests. Path={}", rc.request().path());
+            rc.fail(wrapHttpException(HttpResponseStatus.SERVICE_UNAVAILABLE,

Review Comment:
   Should the status code be a 429 instead? maybe we should follow the implementation of `org.apache.cassandra.sidecar.utils.FileStreamer#acquire` ?



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationDigestHandlerWrapper.java:
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import com.google.inject.Inject;
+import io.vertx.core.Handler;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFileDigestRequest;
+
+import static org.apache.cassandra.sidecar.common.request.LiveMigrationFileDigestRequest.DIGEST_ALGORITHM_PARAM;
+
+/**
+ * Wrapper handler that conditionally delegates to {@link LiveMigrationFileDigestHandler} based on the presence
+ * of the {@link LiveMigrationFileDigestRequest#DIGEST_ALGORITHM_PARAM} query parameter, otherwise
+ * passes control to the next handler in the chain.
+ */
+public class LiveMigrationDigestHandlerWrapper implements Handler<RoutingContext>

Review Comment:
   ```suggestion
   @Singleton
   public class LiveMigrationDigestHandlerWrapper implements Handler<RoutingContext>
   ```



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationFilesVerificationTaskFactory.java:
##########
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import io.vertx.core.Vertx;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+import org.apache.cassandra.sidecar.utils.DigestVerifierFactory;
+import org.apache.cassandra.sidecar.utils.SidecarClientProvider;
+
+/**
+ * Factory for creating {@link LiveMigrationFilesVerificationTask} instances used to verify file digests
+ * between source and destination instances during live migration operation.
+ */
+@Singleton
+public class LiveMigrationFilesVerificationTaskFactory
+{
+    private final Vertx vertx;
+    private final SidecarClientProvider sidecarClientProvider;
+    private final ExecutorPools executorPools;
+    private final DigestVerifierFactory digestVerifierFactory;
+    private final SidecarConfiguration sidecarConfiguration;
+
+    @Inject
+    public LiveMigrationFilesVerificationTaskFactory(Vertx vertx,
+                                                     ExecutorPools executorPools,
+                                                     SidecarConfiguration sidecarConfiguration,
+                                                     SidecarClientProvider sidecarClientProvider,
+                                                     DigestVerifierFactory digestVerifierFactory)
+    {
+        this.vertx = vertx;
+        this.sidecarClientProvider = sidecarClientProvider;
+        this.executorPools = executorPools;
+        this.digestVerifierFactory = digestVerifierFactory;
+        this.sidecarConfiguration = sidecarConfiguration;
+    }
+
+    public LiveMigrationTask<LiveMigrationFilesVerificationResponse> create(String id,
+                                                                            String source,
+                                                                            int port,
+                                                                            LiveMigrationFilesVerificationRequest request,
+                                                                            InstanceMetadata localInstanceMetadata)
+    {
+        return LiveMigrationFilesVerificationTask.builder()
+                                                 .id(id)
+                                                 .source(source)
+                                                 .port(port)
+                                                 .vertx(vertx)
+                                                 .executorPools(executorPools)
+                                                 .sidecarClient(sidecarClientProvider.get())

Review Comment:
   What's the side-effect of calling `sidecarClientProvider.get()`? I don't remember off the top of my head. Can you ensure that calling `sidecarClientProvider.get()` does not create a new client? We should reuse a single client throughout the application.



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCreateFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.json.DecodeException;
+import io.vertx.core.json.Json;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationTaskCreationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationInvalidRequestException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.NoSuchCassandraInstanceException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.ACCEPTED;
+import static io.netty.handler.codec.http.HttpResponseStatus.BAD_REQUEST;
+import static io.netty.handler.codec.http.HttpResponseStatus.CONFLICT;
+import static io.netty.handler.codec.http.HttpResponseStatus.SERVICE_UNAVAILABLE;
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.common.ApiEndpointsV1.LIVE_MIGRATION_FILES_VERIFICATION_TASKS_ROUTE;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * HTTP handler for creating file digest verification tasks during live migration.
+ * Manages concurrent verification tasks per instance and orchestrates the verification process.
+ */
+public class LiveMigrationCreateFilesVerificationTaskHandler extends AbstractHandler<LiveMigrationFilesVerificationRequest> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationCreateFilesVerificationTaskHandler.class);
+
+    private final FilesVerificationTaskManager filesVerificationTaskManager;
+    private final LiveMigrationMap liveMigrationMap;
+
+    /**
+     * Constructs a handler with the provided {@code metadataFetcher}
+     *
+     * @param metadataFetcher the interface to retrieve instance metadata
+     * @param executorPools   the executor pools for blocking executions
+     * @param validator       a validator instance to validate Cassandra-specific input
+     */
+    @Inject
+    protected LiveMigrationCreateFilesVerificationTaskHandler(InstanceMetadataFetcher metadataFetcher,
+                                                              ExecutorPools executorPools,
+                                                              CassandraInputValidator validator,
+                                                              LiveMigrationMap liveMigrationMap,
+                                                              FilesVerificationTaskManager filesVerificationTaskManager)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.liveMigrationMap = liveMigrationMap;
+        this.filesVerificationTaskManager = filesVerificationTaskManager;
+    }
+
+    @Override
+    protected LiveMigrationFilesVerificationRequest extractParamsOrThrow(RoutingContext context)
+    {
+        try
+        {
+            return Json.decodeValue(context.body().buffer(), LiveMigrationFilesVerificationRequest.class);
+        }
+        catch (DecodeException decodeException)
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST,
+                                    "Failed to parse request body, please ensure that the request is valid.",
+                                    decodeException);
+        }
+        catch (IllegalArgumentException e)
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST, e.getMessage(), e);
+        }
+    }
+
+    @Override
+    protected void handleInternal(RoutingContext context,
+                                  HttpServerRequest httpRequest,
+                                  @NotNull String host,
+                                  SocketAddress remoteAddress,
+                                  LiveMigrationFilesVerificationRequest request)
+    {
+        LOGGER.debug("Received files verification request for host {} with maxConcurrency {}",

Review Comment:
   this log entry might be redundant, as we already have it in `org.apache.cassandra.sidecar.handlers.AbstractHandler#handle` where we log the request. I guess we just need to make sure that the `request` has a `toString()` method



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationFileDigestHandler.java:
##########
@@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.Vertx;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.acl.authorization.BasicPermissions;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFileDigestRequest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.handlers.FileStreamHandler;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.DigestAlgorithm;
+import org.apache.cassandra.sidecar.utils.DigestAlgorithmFactory;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.INTERNAL_SERVER_ERROR;
+import static org.apache.cassandra.sidecar.common.request.LiveMigrationFileDigestRequest.DIGEST_ALGORITHM_PARAM;
+import static org.apache.cassandra.sidecar.utils.AsyncFileDigestCalculator.calculateDigest;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler for calculating and returning file digests during live migration.
+ * Supported digest algorithms are determined by {@link DigestAlgorithmFactory} and specified
+ * via the {@link LiveMigrationFileDigestRequest#DIGEST_ALGORITHM_PARAM} query parameter.
+ */
+public class LiveMigrationFileDigestHandler extends AbstractHandler<DigestAlgorithm> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationFileDigestHandler.class);
+
+    private final Vertx vertx;
+    private final DigestAlgorithmFactory digestAlgorithmFactory;
+
+    @Inject
+    public LiveMigrationFileDigestHandler(InstanceMetadataFetcher metadataFetcher,
+                                          ExecutorPools executorPools,
+                                          CassandraInputValidator validator,
+                                          Vertx vertx,
+                                          DigestAlgorithmFactory digestAlgorithmFactory)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.vertx = vertx;
+        this.digestAlgorithmFactory = digestAlgorithmFactory;
+    }
+
+    @Override
+    protected DigestAlgorithm extractParamsOrThrow(RoutingContext context)
+    {
+        String digestAlgorithmParam = getDigestAlgorithmParam(context);
+        try
+        {
+            return digestAlgorithmFactory.getDigestAlgorithm(digestAlgorithmParam, 0);
+        }
+        catch (IllegalArgumentException e)
+        {
+            LOGGER.error("Unexpected error while getting digest algorithm for {}", digestAlgorithmParam, e);
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST, e.getMessage());
+        }
+    }
+
+    @Override
+    protected void handleInternal(RoutingContext context,
+                                  HttpServerRequest httpRequest,
+                                  @NotNull String host,
+                                  SocketAddress remoteAddress,
+                                  DigestAlgorithm digestAlgorithm)
+    {
+        String file = context.get(FileStreamHandler.FILE_PATH_CONTEXT_KEY);
+
+        if (file == null)
+        {
+            LOGGER.error("File path not found in context");
+            context.fail(wrapHttpException(INTERNAL_SERVER_ERROR, "File path not available"));
+            return;
+        }
+        calculateDigest(vertx, file, digestAlgorithm)
+        .onComplete(ar -> {

Review Comment:
   Why don't we re-use the existing machinery for digest verification? it feels like we are rewriting the same work instead of adjusting to the existing implementation.
   
   ```
           DigestVerifier digestVerifier = digestVerifierFactory.verifier(httpRequest.headers());
           digestVerifier.verify(file)
   ````



##########
client-common/src/test/java/org/apache/cassandra/sidecar/common/response/LiveMigrationFilesVerificationResponseTest.java:
##########
@@ -0,0 +1,322 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.common.response;
+
+import org.junit.jupiter.api.Test;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+class LiveMigrationFilesVerificationResponseTest
+{
+    private final ObjectMapper mapper = new ObjectMapper();
+
+    @Test
+    void testSerializationRoundTrip() throws Exception
+    {
+        LiveMigrationFilesVerificationResponse original = new LiveMigrationFilesVerificationResponse(
+        "test-id-123",
+        "MD5",
+        "COMPLETED",
+        "192.168.1.100",
+        9042,
+        5,
+        10,
+        100,
+        15,
+        20,
+        3,
+        85
+        );
+
+        // Serialize to JSON
+        String json = mapper.writeValueAsString(original);
+
+        // Deserialize back to object
+        LiveMigrationFilesVerificationResponse deserialized =
+        mapper.readValue(json, LiveMigrationFilesVerificationResponse.class);
+
+        // Verify all fields match
+        assertThat(deserialized.id()).isEqualTo(original.id());
+        assertThat(deserialized.digestAlgorithm()).isEqualTo(original.digestAlgorithm());
+        assertThat(deserialized.state()).isEqualTo(original.state());
+        assertThat(deserialized.source()).isEqualTo(original.source());
+        assertThat(deserialized.port()).isEqualTo(original.port());
+        assertThat(deserialized.filesNotFoundAtSource()).isEqualTo(original.filesNotFoundAtSource());
+        assertThat(deserialized.filesNotFoundAtDestination()).isEqualTo(original.filesNotFoundAtDestination());
+        assertThat(deserialized.metadataMatched()).isEqualTo(original.metadataMatched());
+        assertThat(deserialized.metadataMismatches()).isEqualTo(original.metadataMismatches());
+        assertThat(deserialized.digestMismatches()).isEqualTo(original.digestMismatches());
+        assertThat(deserialized.digestVerificationFailures()).isEqualTo(original.digestVerificationFailures());
+        assertThat(deserialized.filesMatched()).isEqualTo(original.filesMatched());
+        assertThat(deserialized.isVerificationSuccessful()).isEqualTo(original.isVerificationSuccessful());

Review Comment:
   Let's move to a helper method and reuse for other tests.



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationFileDigestHandler.java:
##########
@@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.Vertx;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.acl.authorization.BasicPermissions;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFileDigestRequest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.handlers.FileStreamHandler;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.DigestAlgorithm;
+import org.apache.cassandra.sidecar.utils.DigestAlgorithmFactory;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.INTERNAL_SERVER_ERROR;
+import static org.apache.cassandra.sidecar.common.request.LiveMigrationFileDigestRequest.DIGEST_ALGORITHM_PARAM;
+import static org.apache.cassandra.sidecar.utils.AsyncFileDigestCalculator.calculateDigest;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler for calculating and returning file digests during live migration.
+ * Supported digest algorithms are determined by {@link DigestAlgorithmFactory} and specified
+ * via the {@link LiveMigrationFileDigestRequest#DIGEST_ALGORITHM_PARAM} query parameter.
+ */
+public class LiveMigrationFileDigestHandler extends AbstractHandler<DigestAlgorithm> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationFileDigestHandler.class);
+
+    private final Vertx vertx;
+    private final DigestAlgorithmFactory digestAlgorithmFactory;
+
+    @Inject
+    public LiveMigrationFileDigestHandler(InstanceMetadataFetcher metadataFetcher,
+                                          ExecutorPools executorPools,
+                                          CassandraInputValidator validator,
+                                          Vertx vertx,
+                                          DigestAlgorithmFactory digestAlgorithmFactory)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.vertx = vertx;
+        this.digestAlgorithmFactory = digestAlgorithmFactory;
+    }
+
+    @Override
+    protected DigestAlgorithm extractParamsOrThrow(RoutingContext context)
+    {
+        String digestAlgorithmParam = getDigestAlgorithmParam(context);
+        try
+        {
+            return digestAlgorithmFactory.getDigestAlgorithm(digestAlgorithmParam, 0);
+        }
+        catch (IllegalArgumentException e)
+        {
+            LOGGER.error("Unexpected error while getting digest algorithm for {}", digestAlgorithmParam, e);
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST, e.getMessage());
+        }

Review Comment:
   I think it's redundant to handle the `IllegalArgumentException` here. This will be correctly handled on an exception here: `org.apache.cassandra.sidecar.handlers.AbstractHandler#processFailure`
   ```suggestion
           String digestAlgorithmParam = getDigestAlgorithmParam(context);
           return digestAlgorithmFactory.getDigestAlgorithm(digestAlgorithmParam, 0);
   ```



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationTaskManager.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+
+import com.google.common.annotations.VisibleForTesting;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import org.apache.cassandra.sidecar.cluster.InstancesMetadata;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Centralized singleton manager for all live migration tasks across Cassandra instances.
+ * Enforces the constraint that only one {@link LiveMigrationTask} (of any type) can be active per instance at a time,
+ * preventing concurrent data copy and files verification operations that could lead to resource conflicts or data integrity issues.
+ * This singleton is shared across {@link DataCopyTaskManager} and {@link FilesVerificationTaskManager} to ensure
+ * proper coordination and mutual exclusion of tasks per instance.
+ */
+@Singleton
+public class LiveMigrationTaskManager
+{
+    @VisibleForTesting
+    final ConcurrentHashMap<Integer, LiveMigrationTask<?>> currentTasks = new ConcurrentHashMap<>();
+
+    private final InstancesMetadata instancesMetadata;
+
+    @Inject
+    public LiveMigrationTaskManager(InstancesMetadata instancesMetadata)
+    {
+        this.instancesMetadata = instancesMetadata;
+    }
+
+    /**
+     * Attempts to submit a new task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time.
+     *
+     * @param instanceId the instance ID
+     * @param newTask    the task to submit
+     * @return true if the task was accepted, false if another task is already in progress
+     */
+    public boolean submitTask(int instanceId, LiveMigrationTask<?> newTask)
+    {
+        return currentTasks.compute(instanceId, (integer, taskInMap) -> {

Review Comment:
   NIT
   ```suggestion
           return currentTasks.compute(instanceId, (ignored, taskInMap) -> {
   ```



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationTaskManager.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+
+import com.google.common.annotations.VisibleForTesting;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import org.apache.cassandra.sidecar.cluster.InstancesMetadata;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Centralized singleton manager for all live migration tasks across Cassandra instances.
+ * Enforces the constraint that only one {@link LiveMigrationTask} (of any type) can be active per instance at a time,
+ * preventing concurrent data copy and files verification operations that could lead to resource conflicts or data integrity issues.
+ * This singleton is shared across {@link DataCopyTaskManager} and {@link FilesVerificationTaskManager} to ensure
+ * proper coordination and mutual exclusion of tasks per instance.
+ */
+@Singleton
+public class LiveMigrationTaskManager
+{
+    @VisibleForTesting
+    final ConcurrentHashMap<Integer, LiveMigrationTask<?>> currentTasks = new ConcurrentHashMap<>();
+
+    private final InstancesMetadata instancesMetadata;
+
+    @Inject
+    public LiveMigrationTaskManager(InstancesMetadata instancesMetadata)
+    {
+        this.instancesMetadata = instancesMetadata;
+    }
+
+    /**
+     * Attempts to submit a new task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time.
+     *
+     * @param instanceId the instance ID
+     * @param newTask    the task to submit
+     * @return true if the task was accepted, false if another task is already in progress
+     */
+    public boolean submitTask(int instanceId, LiveMigrationTask<?> newTask)
+    {
+        return currentTasks.compute(instanceId, (integer, taskInMap) -> {
+            if (taskInMap == null)
+            {
+                return newTask;
+            }
+
+            if (!taskInMap.isCompleted())
+            {
+                // Reject new task if existing task is still in progress
+                return taskInMap;
+            }
+            else
+            {
+                // Accept new task if existing task has completed
+                return newTask;
+            }
+        }) == newTask;
+    }
+
+    /**
+     * Returns all live migration tasks for given currentHost.
+     * This includes both active and completed tasks that haven't been replaced.
+     *
+     * @param currentHost the host where sidecar is running
+     * @return list containing at most one task (empty if no task has ever been submitted for this host)
+     */
+    @SuppressWarnings("ConstantValue")
+    public List<LiveMigrationTask<?>> getAllTasks(@NotNull String currentHost)
+    {
+        InstanceMetadata localInstance = instancesMetadata.instanceFromHost(currentHost);
+        if (localInstance == null)
+        {
+            throw new IllegalStateException("No instance found for host: " + currentHost);
+        }
+
+        LiveMigrationTask<?> task = currentTasks.get(localInstance.id());
+        return task == null ? Collections.emptyList() : Collections.singletonList(task);
+    }
+
+    /**
+     * Returns the live migration task with the specified task ID.
+     *
+     * @param taskId      ID of the task to retrieve
+     * @param currentHost the host where sidecar is running
+     * @return the LiveMigrationTask matching the given taskId
+     * @throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException if no task found with the given ID
+     */
+    public LiveMigrationTask<?> getTask(@NotNull String taskId,
+                                        @NotNull String currentHost) throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException
+    {
+        return getLiveMigrationTask(taskId, currentHost);
+    }
+
+    /**
+     * Cancels the live migration task with the specified task ID.
+     *
+     * @param taskId      ID of the task to cancel
+     * @param currentHost the host where sidecar is running
+     * @return the cancelled LiveMigrationTask
+     * @throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException if no task found with the given ID
+     */
+    public LiveMigrationTask<?> cancelTask(@NotNull String taskId,
+                                           @NotNull String currentHost) throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException
+    {
+        LiveMigrationTask<?> taskInProgress = getLiveMigrationTask(taskId, currentHost);
+
+        // Cancelling the task
+        taskInProgress.cancel();
+
+        return taskInProgress;
+    }
+
+    @SuppressWarnings("ConstantValue")
+    private LiveMigrationTask<?> getLiveMigrationTask(@NotNull String taskId, @NotNull String currentHost)
+    {
+        InstanceMetadata localInstance = instancesMetadata.instanceFromHost(currentHost);
+        if (localInstance == null)
+        {
+            throw new IllegalStateException("No instance found for host: " + currentHost);
+        }

Review Comment:
   not necessary to do a null check here
   ```suggestion
   ```



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationGetAllFilesVerificationTasksHandler.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import com.google.inject.Inject;
+import io.vertx.core.Handler;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.livemigration.LiveMigrationTask;
+
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.handlers.AbstractHandler.extractHostAddressWithoutPort;
+
+/**
+ * Handler for retrieving all active live migration files verification tasks.
+ * Returns a list of all verification tasks running on the local host.
+ */
+public class LiveMigrationGetAllFilesVerificationTasksHandler implements Handler<RoutingContext>, AccessProtected

Review Comment:
   ```suggestion
   @Singleton
   public class LiveMigrationGetAllFilesVerificationTasksHandler implements Handler<RoutingContext>, AccessProtected
   ```



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationGetFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.livemigration.LiveMigrationTask;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler for retrieving a specific live migration files verification task by task ID.
+ * Returns the task details if found, or a 404 error if the task does not exist on the specified host.
+ */
+public class LiveMigrationGetFilesVerificationTaskHandler extends AbstractHandler<String> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationGetFilesVerificationTaskHandler.class);
+
+    private final FilesVerificationTaskManager taskManager;
+
+    /**
+     * Constructs a handler with the provided {@code metadataFetcher}
+     *
+     * @param metadataFetcher the interface to retrieve instance metadata
+     * @param executorPools   the executor pools for blocking executions
+     * @param validator       a validator instance to validate Cassandra-specific input
+     */
+    @Inject
+    public LiveMigrationGetFilesVerificationTaskHandler(InstanceMetadataFetcher metadataFetcher,
+                                                        ExecutorPools executorPools,
+                                                        CassandraInputValidator validator,
+                                                        FilesVerificationTaskManager taskManager)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.taskManager = taskManager;
+    }
+
+    @Override
+    protected String extractParamsOrThrow(RoutingContext context)
+    {
+        String taskId = context.pathParam("taskId");
+        if (taskId == null || taskId.isEmpty())
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST, "taskId is required");
+        }
+
+        return taskId;
+    }
+
+    @Override
+    protected void handleInternal(RoutingContext context,
+                                  HttpServerRequest httpRequest,
+                                  @NotNull String host,
+                                  SocketAddress remoteAddress,
+                                  String taskId)
+    {
+        try
+        {
+            LiveMigrationTask<LiveMigrationFilesVerificationResponse> task = taskManager.getTask(taskId, host);
+            LOGGER.info("Found live migration task with taskId={} on host={}", taskId, host);

Review Comment:
   seems too much, can we lower to debug?



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationTaskManager.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+
+import com.google.common.annotations.VisibleForTesting;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import org.apache.cassandra.sidecar.cluster.InstancesMetadata;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Centralized singleton manager for all live migration tasks across Cassandra instances.
+ * Enforces the constraint that only one {@link LiveMigrationTask} (of any type) can be active per instance at a time,
+ * preventing concurrent data copy and files verification operations that could lead to resource conflicts or data integrity issues.
+ * This singleton is shared across {@link DataCopyTaskManager} and {@link FilesVerificationTaskManager} to ensure
+ * proper coordination and mutual exclusion of tasks per instance.
+ */
+@Singleton
+public class LiveMigrationTaskManager
+{
+    @VisibleForTesting
+    final ConcurrentHashMap<Integer, LiveMigrationTask<?>> currentTasks = new ConcurrentHashMap<>();
+
+    private final InstancesMetadata instancesMetadata;
+
+    @Inject
+    public LiveMigrationTaskManager(InstancesMetadata instancesMetadata)
+    {
+        this.instancesMetadata = instancesMetadata;
+    }
+
+    /**
+     * Attempts to submit a new task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time.
+     *
+     * @param instanceId the instance ID
+     * @param newTask    the task to submit
+     * @return true if the task was accepted, false if another task is already in progress
+     */
+    public boolean submitTask(int instanceId, LiveMigrationTask<?> newTask)
+    {
+        return currentTasks.compute(instanceId, (integer, taskInMap) -> {
+            if (taskInMap == null)
+            {
+                return newTask;
+            }
+
+            if (!taskInMap.isCompleted())
+            {
+                // Reject new task if existing task is still in progress
+                return taskInMap;
+            }
+            else
+            {
+                // Accept new task if existing task has completed
+                return newTask;
+            }
+        }) == newTask;
+    }
+
+    /**
+     * Returns all live migration tasks for given currentHost.
+     * This includes both active and completed tasks that haven't been replaced.
+     *
+     * @param currentHost the host where sidecar is running
+     * @return list containing at most one task (empty if no task has ever been submitted for this host)
+     */
+    @SuppressWarnings("ConstantValue")
+    public List<LiveMigrationTask<?>> getAllTasks(@NotNull String currentHost)
+    {
+        InstanceMetadata localInstance = instancesMetadata.instanceFromHost(currentHost);
+        if (localInstance == null)
+        {
+            throw new IllegalStateException("No instance found for host: " + currentHost);
+        }
+

Review Comment:
   this check is not required. instanceFromHost returns non-null results



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSSIDECAR-226 Adding endpoint for verifying files post data copy during live migration [cassandra-sidecar]

Posted by "nvharikrishna (via GitHub)" <gi...@apache.org>.
nvharikrishna commented on code in PR #309:
URL: https://github.com/apache/cassandra-sidecar/pull/309#discussion_r2891845780


##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationFilesVerificationTask.java:
##########
@@ -0,0 +1,643 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.Callable;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.vertx.core.Future;
+import io.vertx.core.MultiMap;
+import io.vertx.core.Promise;
+import io.vertx.core.Vertx;
+import org.apache.cassandra.sidecar.client.SidecarClient;
+import org.apache.cassandra.sidecar.client.SidecarInstanceImpl;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.DataObjectBuilder;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.request.data.Digest;
+import org.apache.cassandra.sidecar.common.request.data.MD5Digest;
+import org.apache.cassandra.sidecar.common.request.data.XXHash32Digest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.common.response.InstanceFileInfo;
+import org.apache.cassandra.sidecar.common.response.InstanceFilesListResponse;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.AsyncConcurrentTaskExecutor;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.DigestMismatchException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.FileVerificationFailureException;
+import org.apache.cassandra.sidecar.utils.DigestVerifierFactory;
+import org.jetbrains.annotations.NotNull;
+import org.jetbrains.annotations.VisibleForTesting;
+
+import static org.apache.cassandra.sidecar.common.response.InstanceFileInfo.FileType.DIRECTORY;
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationInstanceMetadataUtil.localPath;
+
+/**
+ * Verifies file integrity between source and destination instances during live migration
+ * by comparing file lists and validating file digests using configurable digest algorithms.
+ *
+ * <p>The verification process consists of three stages:
+ * <ol>
+ *   <li>Fetch file lists from both source and destination instances concurrently</li>
+ *   <li>Compare file metadata (size, type, modification time) - fails fast on mismatches</li>
+ *   <li>Verify cryptographic digests (MD5 or XXHash32) with configurable concurrency</li>
+ * </ol>
+ *
+ * <p>The task supports cancellation, and tracks detailed metrics including
+ * metadata matches/mismatches, digest verification results, and failure counts.
+ */
+public class LiveMigrationFilesVerificationTask implements LiveMigrationTask<LiveMigrationFilesVerificationResponse>
+{
+    public static final String FILES_VERIFICATION_TASK_TYPE = "files-verification-task";
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationFilesVerificationTask.class);
+    private final Vertx vertx;
+    private final String id;
+    private final String source;
+    private final int port;
+    // Instance metadata of current instance
+    private final InstanceMetadata instanceMetadata;
+    private final ExecutorPools executorPools;
+    private final LiveMigrationFilesVerificationRequest request;
+    private final SidecarClient sidecarClient;
+    private final LiveMigrationConfiguration liveMigrationConfiguration;
+    private final DigestVerifierFactory digestVerifierFactory;
+    private final AtomicReference<State> state;
+    private final Promise<Void> completionPromise = Promise.promise();
+    private final AtomicInteger filesNotFoundAtSource = new AtomicInteger(0);
+    private final AtomicInteger filesNotFoundAtDestination = new AtomicInteger(0);
+    private final AtomicInteger metadataMatched = new AtomicInteger(0);
+    private final AtomicInteger metadataMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestVerificationFailures = new AtomicInteger(0);
+    private final AtomicInteger filesMatched = new AtomicInteger(0);
+    private final String logPrefix;
+    private final AtomicBoolean cancelled = new AtomicBoolean(false);
+    private volatile AsyncConcurrentTaskExecutor<String> asyncConcurrentTaskExecutor;
+
+    @VisibleForTesting
+    protected LiveMigrationFilesVerificationTask(Builder builder)
+    {
+        this.vertx = builder.vertx;
+        this.id = builder.id;
+        this.source = builder.source;
+        this.port = builder.port;
+        this.instanceMetadata = builder.instanceMetadata;
+        this.executorPools = builder.executorPools;
+        this.request = builder.request;
+        this.sidecarClient = builder.sidecarClient;
+        this.liveMigrationConfiguration = builder.liveMigrationConfiguration;
+        this.digestVerifierFactory = builder.digestVerifierFactory;
+        this.logPrefix = "fileVerifyTaskId=" + id;
+        this.state = new AtomicReference<>(State.NOT_STARTED);
+    }
+
+    public static Builder builder()
+    {
+        return new Builder();
+    }
+
+    public Future<Void> validate()
+    {
+        return abortIfCancelled(new Object())
+               // Get files info from both source and destination
+               .compose(v -> Future.join(listFilesInLocal(), fetchSourceFileList()))
+
+               // Compare the files information before calculating digests
+               .compose(this::abortIfCancelled)
+               .compose(cf -> compareFilesMeta(cf.resultAt(0), cf.resultAt(1)))
+
+               // Create digest comparison tasks
+               .compose(this::abortIfCancelled)
+               .compose(this::getDigestComparisonTasks)
+
+               // Compare digests
+               .compose(this::abortIfCancelled)
+               .compose(this::compareDigests)
+
+               .andThen(ar -> {
+                   if (ar.failed())
+                   {
+                       LOGGER.error("{} Files verification task failed during validation", logPrefix, ar.cause());
+                       // Fail the task if it is in IN_PROGRESS state.
+                       state.compareAndSet(State.IN_PROGRESS, State.FAILED);
+                   }
+                   else
+                   {
+                       State currentState = state.compareAndExchange(State.IN_PROGRESS, State.COMPLETED);
+                       if (currentState == State.CANCELLED)
+                       {
+                           LOGGER.debug("{} Task was cancelled before completion", logPrefix);
+                       }
+                       else if (currentState != State.IN_PROGRESS)
+                       {
+                           LOGGER.warn("{} Unexpected state transition failure. State was {}", logPrefix, currentState);
+                       }
+                       else
+                       {
+                           LOGGER.info("{} Files verification task completed successfully", logPrefix);
+                       }
+                   }
+               });
+    }
+
+    private Future<List<InstanceFileInfo>> listFilesInLocal()
+    {
+        return executorPools.internal()
+                            .executeBlocking(() -> new CassandraInstanceFilesImpl(instanceMetadata,
+                                                                                  liveMigrationConfiguration)
+                                                   .files());
+    }
+
+    private Future<InstanceFilesListResponse> fetchSourceFileList()
+    {
+        return Future.fromCompletionStage(
+                     sidecarClient.liveMigrationListInstanceFilesAsync(new SidecarInstanceImpl(source, port)))
+                     .onFailure(cause -> LOGGER.error("{} Failed to obtain list of files from source {}:{}",
+                                                      logPrefix, source, port, cause));
+    }
+
+    private @NotNull Future<List<InstanceFileInfo>> compareFilesMeta(List<InstanceFileInfo> localFiles,
+                                                                     InstanceFilesListResponse sourceFiles)
+    {
+        Map<String, InstanceFileInfo> filesAtLocal =
+        localFiles.stream()
+                  .collect(Collectors.toMap(fileInfo -> fileInfo.fileUrl, fileInfo -> fileInfo));
+
+        Map<String, InstanceFileInfo> filesAtSource =
+        sourceFiles.getFiles().stream()
+                   .collect(Collectors.toMap(fileInfo -> fileInfo.fileUrl, fileInfo -> fileInfo));
+
+        for (Map.Entry<String, InstanceFileInfo> localFileEntry : filesAtLocal.entrySet())
+        {
+            String fileUrl = localFileEntry.getKey();
+            InstanceFileInfo localFile = localFileEntry.getValue();
+            if (filesAtSource.containsKey(fileUrl))
+            {
+                InstanceFileInfo sourceFile = filesAtSource.get(fileUrl);
+                // compare files metadata
+                if (localFile.equals(sourceFile) ||
+                    (localFile.fileType == DIRECTORY && sourceFile.fileType == DIRECTORY))
+                {
+                    // File info matches if either of them are directories or
+                    // their size, last modified time etc... matches
+                    LOGGER.debug("{} {} file metadata matched with source", logPrefix, fileUrl);
+                    metadataMatched.incrementAndGet();
+                }
+                else
+                {
+                    // Files metadata did not match
+                    metadataMismatches.incrementAndGet();
+                    LOGGER.error("{} {} did not match with source. Local file info={}, source file info={}",
+                                 logPrefix, fileUrl, localFile, sourceFile);
+                }
+
+                // done with processing current local file, remove it
+                filesAtSource.remove(fileUrl);
+            }
+            else
+            {
+                // File is not available at source, report it
+                LOGGER.error("{} File {} exists at destination but not found at source", logPrefix, fileUrl);
+                filesNotFoundAtSource.incrementAndGet();
+            }
+        }
+
+        for (Map.Entry<String, InstanceFileInfo> sourceFileEntry : filesAtSource.entrySet())
+        {
+            // Remaining files are not present at destination, report error for them
+            LOGGER.error("{} File {} exists at source but not found at destination", logPrefix, sourceFileEntry.getKey());
+            filesNotFoundAtDestination.incrementAndGet();
+        }
+
+        if (filesNotFoundAtDestination.get() > 0
+            || filesNotFoundAtSource.get() > 0
+            || metadataMismatches.get() > 0)
+        {
+            FileVerificationFailureException exception =
+            new FileVerificationFailureException("Files list did not match between source and destination");
+
+            return Future.failedFuture(exception);
+        }
+
+        return Future.succeededFuture(localFiles);
+    }
+
+    private @NotNull Future<List<Future<String>>> getDigestComparisonTasks(List<InstanceFileInfo> localFiles)
+    {
+        // now verify digests of each local file with source file
+        List<Callable<Future<String>>> tasks = new ArrayList<>(localFiles.size());
+        localFiles.stream()
+                  .filter(fileInfo -> fileInfo.fileType != DIRECTORY)
+                  .forEach(fileInfo -> tasks.add(() -> verifyDigest(fileInfo)));
+        asyncConcurrentTaskExecutor = new AsyncConcurrentTaskExecutor<>(vertx, tasks, request.maxConcurrency());
+        List<Future<String>> verifyTaskResults = asyncConcurrentTaskExecutor.start();
+        return Future.join(verifyTaskResults)
+                     .transform(ar -> Future.succeededFuture(verifyTaskResults));
+    }
+
+    private Future<Void> compareDigests(List<Future<String>> futures)
+    {
+        boolean verificationFailed = false;
+        for (Future<String> future : futures)
+        {
+            if (future.failed())
+            {
+                verificationFailed = true;
+                Throwable cause = future.cause();
+                if (cause instanceof DigestMismatchException)
+                {
+                    DigestMismatchException ex = (DigestMismatchException) cause;
+                    LOGGER.error("{} File digests did not match for {}.", logPrefix, ex.fileUrl(), cause);
+                    digestMismatches.incrementAndGet();
+                }
+                else
+                {
+                    LOGGER.error("{} Failed to verify digest for file. Check previous logs for file details",
+                                 logPrefix, cause);
+                    digestVerificationFailures.incrementAndGet();
+                }
+                continue;
+            }
+
+            // Digest verification succeeded
+            filesMatched.incrementAndGet();
+        }
+
+        if (verificationFailed)
+        {
+            return Future.failedFuture(
+            new FileVerificationFailureException("File digests did not match between source and destination"));
+        }
+        else
+        {
+            return Future.succeededFuture();
+        }
+    }
+
+    @VisibleForTesting
+    Future<String> verifyDigest(InstanceFileInfo fileInfo)
+    {
+        return getSourceFileDigest(fileInfo)
+               .compose(digest -> {
+                   String path = localPath(fileInfo.fileUrl, instanceMetadata).toAbsolutePath().toString();
+                   return digestVerifierFactory.verifier(MultiMap.caseInsensitiveMultiMap().addAll(digest.headers()))
+                                               .verify(path)
+                                               .compose(verified -> Future.succeededFuture(path))
+                                               .recover(cause -> Future.failedFuture(
+                                               new DigestMismatchException(path, fileInfo.fileUrl, cause)));
+               })
+               .onSuccess(filePath -> LOGGER.debug("{} Verified file {}", logPrefix, fileInfo.fileUrl))
+               .onFailure(cause -> LOGGER.error("{} Failed to verify file {}", logPrefix, fileInfo.fileUrl, cause));
+    }
+
+    private Future<Digest> getSourceFileDigest(InstanceFileInfo fileInfo)
+    {
+        return Future.fromCompletionStage(sidecarClient.liveMigrationFileDigestAsync(new SidecarInstanceImpl(source, port),
+                                                                                     fileInfo.fileUrl,
+                                                                                     request.digestAlgorithm()))
+                     .compose(this::toDigest);
+    }

Review Comment:
   The concurrency limit used here (`sidecarConfiguration.liveMigrationConfiguration().maxConcurrentFileRequests()`) is global across all clients, not per-client. A client sending its very first request can get rejected because other clients have exhausted the pool.
   
   429 means "the user has sent too many requests" — implying the specific client is at fault, which I feel misleading here. 503 could also mean "the server is unable to handle the request due to a temporary overload" - which I think describes the situation in this case.
   
   If this were a per-client throttle, 429 would be correct. Since maxConcurrentFileRequests is a global limit protecting the source node, I feel 503 is more accurate.
   
   Happy to change it to 429 if you feel differently though! Open to any other suggestions as well.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSSIDECAR-226 Adding endpoint for verifying files post data copy during live migration [cassandra-sidecar]

Posted by "nvharikrishna (via GitHub)" <gi...@apache.org>.
nvharikrishna commented on code in PR #309:
URL: https://github.com/apache/cassandra-sidecar/pull/309#discussion_r3180505098


##########
client-common/src/test/java/org/apache/cassandra/sidecar/common/response/LiveMigrationFilesVerificationResponseTest.java:
##########
@@ -0,0 +1,322 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.common.response;
+
+import org.junit.jupiter.api.Test;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+class LiveMigrationFilesVerificationResponseTest
+{
+    private final ObjectMapper mapper = new ObjectMapper();
+
+    @Test
+    void testSerializationRoundTrip() throws Exception
+    {
+        LiveMigrationFilesVerificationResponse original = new LiveMigrationFilesVerificationResponse(
+        "test-id-123",
+        "MD5",
+        "COMPLETED",
+        "192.168.1.100",
+        9042,
+        5,
+        10,
+        100,
+        15,
+        20,
+        3,
+        85
+        );
+
+        // Serialize to JSON
+        String json = mapper.writeValueAsString(original);
+
+        // Deserialize back to object
+        LiveMigrationFilesVerificationResponse deserialized =
+        mapper.readValue(json, LiveMigrationFilesVerificationResponse.class);
+
+        // Verify all fields match
+        assertThat(deserialized.id()).isEqualTo(original.id());
+        assertThat(deserialized.digestAlgorithm()).isEqualTo(original.digestAlgorithm());
+        assertThat(deserialized.state()).isEqualTo(original.state());
+        assertThat(deserialized.source()).isEqualTo(original.source());
+        assertThat(deserialized.port()).isEqualTo(original.port());
+        assertThat(deserialized.filesNotFoundAtSource()).isEqualTo(original.filesNotFoundAtSource());
+        assertThat(deserialized.filesNotFoundAtDestination()).isEqualTo(original.filesNotFoundAtDestination());
+        assertThat(deserialized.metadataMatched()).isEqualTo(original.metadataMatched());
+        assertThat(deserialized.metadataMismatches()).isEqualTo(original.metadataMismatches());
+        assertThat(deserialized.digestMismatches()).isEqualTo(original.digestMismatches());
+        assertThat(deserialized.digestVerificationFailures()).isEqualTo(original.digestVerificationFailures());
+        assertThat(deserialized.filesMatched()).isEqualTo(original.filesMatched());
+        assertThat(deserialized.isVerificationSuccessful()).isEqualTo(original.isVerificationSuccessful());

Review Comment:
   Now it appears only once after using parameterised tests.



##########
client-common/src/main/java/org/apache/cassandra/sidecar/common/response/DigestResponse.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.common.response;
+
+import java.util.Objects;
+
+import com.fasterxml.jackson.annotation.JsonCreator;
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import org.apache.cassandra.sidecar.common.request.data.Digest;
+import org.apache.cassandra.sidecar.common.request.data.MD5Digest;
+import org.apache.cassandra.sidecar.common.request.data.XXHash32Digest;
+
+/**
+ * Response object containing a cryptographic digest value for file verification purposes.
+ */
+public class DigestResponse
+{
+    @JsonProperty("digest")
+    public final String digest;
+    @JsonProperty("digestAlgorithm")
+    public final String digestAlgorithm;
+
+    @JsonCreator
+    public DigestResponse(@JsonProperty("digest") String digest,
+                          @JsonProperty("digestAlgorithm") String digestAlgorithm)
+    {
+        Objects.requireNonNull(digest, "digest is required");
+        Objects.requireNonNull(digestAlgorithm, "digestAlgorithm is required");
+        this.digest = digest;
+        this.digestAlgorithm = digestAlgorithm;
+    }

Review Comment:
   Done



##########
client-common/src/main/java/org/apache/cassandra/sidecar/common/response/LiveMigrationFilesVerificationResponse.java:
##########
@@ -0,0 +1,186 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.common.response;
+
+import java.util.Objects;
+
+import com.fasterxml.jackson.annotation.JsonCreator;
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+/**
+ * Response object for live migration file verification operations, containing statistics about
+ * files not found at source/target and digest mismatches during verification.
+ */
+@JsonIgnoreProperties(ignoreUnknown = true)
+public class LiveMigrationFilesVerificationResponse
+{
+    private final String id;
+    private final String state;
+    private final String source;
+    private final int port;
+    private final int filesNotFoundAtSource;
+    private final int filesNotFoundAtDestination;
+    private final int metadataMatched;
+    private final int metadataMismatches;
+    private final int digestMismatches;
+    private final int digestVerificationFailures;
+    private final int filesMatched;
+    private final String digestAlgorithm;
+
+    @JsonCreator
+    public LiveMigrationFilesVerificationResponse(@JsonProperty("id") String id,
+                                                  @JsonProperty("digestAlgorithm") String digestAlgorithm,
+                                                  @JsonProperty("state") String state,
+                                                  @JsonProperty("source") String source,
+                                                  @JsonProperty("port") int port,
+                                                  @JsonProperty("filesNotFoundAtSource") int filesNotFoundAtSource,
+                                                  @JsonProperty("filesNotFoundAtDestination") int filesNotFoundAtDestination,
+                                                  @JsonProperty("metadataMatched") int metadataMatched,
+                                                  @JsonProperty("metadataMismatches") int metadataMismatches,
+                                                  @JsonProperty("digestMismatches") int digestMismatches,
+                                                  @JsonProperty("digestVerificationFailures") int digestVerificationFailures,
+                                                  @JsonProperty("filesMatched") int filesMatched)
+    {
+        Objects.requireNonNull(id, "id of files verification task must be specified");
+        Objects.requireNonNull(state, "state of files verification task must be specified");
+        this.id = id;
+        this.digestAlgorithm = digestAlgorithm;
+        this.state = state;

Review Comment:
   done



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCancelFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.livemigration.LiveMigrationTask;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler for canceling an active live migration files verification task.
+ * Accepts a task ID and cancels the corresponding verification task on the specified host.
+ */
+public class LiveMigrationCancelFilesVerificationTaskHandler extends AbstractHandler<String> implements AccessProtected

Review Comment:
   done



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationConcurrencyLimitHandler.java:
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.Handler;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.concurrent.ConcurrencyLimiter;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler that enforces concurrency limits for live migration file operations.
+ * Returns HTTP 503 (SERVICE_UNAVAILABLE) when the maximum concurrent file requests limit is exceeded.
+ */
+@Singleton
+public class LiveMigrationConcurrencyLimitHandler implements Handler<RoutingContext>
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationConcurrencyLimitHandler.class);
+
+    private final ConcurrencyLimiter concurrencyLimiter;
+
+    @Inject
+    public LiveMigrationConcurrencyLimitHandler(SidecarConfiguration sidecarConfiguration)
+    {
+        this.concurrencyLimiter =
+        new ConcurrencyLimiter(() -> sidecarConfiguration.liveMigrationConfiguration().maxConcurrentFileRequests());
+    }
+
+    @Override
+    public void handle(RoutingContext rc)
+    {
+        if (!concurrencyLimiter.tryAcquire())
+        {
+            LOGGER.warn("Too many concurrent live migration file requests. Path={}", rc.request().path());
+            rc.fail(wrapHttpException(HttpResponseStatus.SERVICE_UNAVAILABLE,

Review Comment:
   It is discussed here: https://github.com/apache/cassandra-sidecar/pull/309#discussion_r2868621502



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationFileDigestHandler.java:
##########
@@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.Vertx;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.acl.authorization.BasicPermissions;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFileDigestRequest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.handlers.FileStreamHandler;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.DigestAlgorithm;
+import org.apache.cassandra.sidecar.utils.DigestAlgorithmFactory;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.INTERNAL_SERVER_ERROR;
+import static org.apache.cassandra.sidecar.common.request.LiveMigrationFileDigestRequest.DIGEST_ALGORITHM_PARAM;
+import static org.apache.cassandra.sidecar.utils.AsyncFileDigestCalculator.calculateDigest;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler for calculating and returning file digests during live migration.
+ * Supported digest algorithms are determined by {@link DigestAlgorithmFactory} and specified
+ * via the {@link LiveMigrationFileDigestRequest#DIGEST_ALGORITHM_PARAM} query parameter.
+ */
+public class LiveMigrationFileDigestHandler extends AbstractHandler<DigestAlgorithm> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationFileDigestHandler.class);
+
+    private final Vertx vertx;
+    private final DigestAlgorithmFactory digestAlgorithmFactory;
+
+    @Inject
+    public LiveMigrationFileDigestHandler(InstanceMetadataFetcher metadataFetcher,
+                                          ExecutorPools executorPools,
+                                          CassandraInputValidator validator,
+                                          Vertx vertx,
+                                          DigestAlgorithmFactory digestAlgorithmFactory)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.vertx = vertx;
+        this.digestAlgorithmFactory = digestAlgorithmFactory;
+    }
+
+    @Override
+    protected DigestAlgorithm extractParamsOrThrow(RoutingContext context)
+    {
+        String digestAlgorithmParam = getDigestAlgorithmParam(context);
+        try
+        {
+            return digestAlgorithmFactory.getDigestAlgorithm(digestAlgorithmParam, 0);
+        }
+        catch (IllegalArgumentException e)
+        {
+            LOGGER.error("Unexpected error while getting digest algorithm for {}", digestAlgorithmParam, e);
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST, e.getMessage());
+        }

Review Comment:
   Done



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationTaskManager.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+
+import com.google.common.annotations.VisibleForTesting;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import org.apache.cassandra.sidecar.cluster.InstancesMetadata;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Centralized singleton manager for all live migration tasks across Cassandra instances.
+ * Enforces the constraint that only one {@link LiveMigrationTask} (of any type) can be active per instance at a time,
+ * preventing concurrent data copy and files verification operations that could lead to resource conflicts or data integrity issues.
+ * This singleton is shared across {@link DataCopyTaskManager} and {@link FilesVerificationTaskManager} to ensure
+ * proper coordination and mutual exclusion of tasks per instance.
+ */
+@Singleton
+public class LiveMigrationTaskManager
+{
+    @VisibleForTesting
+    final ConcurrentHashMap<Integer, LiveMigrationTask<?>> currentTasks = new ConcurrentHashMap<>();
+
+    private final InstancesMetadata instancesMetadata;
+
+    @Inject
+    public LiveMigrationTaskManager(InstancesMetadata instancesMetadata)
+    {
+        this.instancesMetadata = instancesMetadata;
+    }
+
+    /**
+     * Attempts to submit a new task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time.
+     *
+     * @param instanceId the instance ID
+     * @param newTask    the task to submit
+     * @return true if the task was accepted, false if another task is already in progress
+     */
+    public boolean submitTask(int instanceId, LiveMigrationTask<?> newTask)
+    {
+        return currentTasks.compute(instanceId, (integer, taskInMap) -> {
+            if (taskInMap == null)
+            {
+                return newTask;
+            }
+
+            if (!taskInMap.isCompleted())
+            {
+                // Reject new task if existing task is still in progress
+                return taskInMap;
+            }
+            else
+            {
+                // Accept new task if existing task has completed
+                return newTask;
+            }
+        }) == newTask;
+    }
+
+    /**
+     * Returns all live migration tasks for given currentHost.
+     * This includes both active and completed tasks that haven't been replaced.
+     *
+     * @param currentHost the host where sidecar is running
+     * @return list containing at most one task (empty if no task has ever been submitted for this host)
+     */
+    @SuppressWarnings("ConstantValue")
+    public List<LiveMigrationTask<?>> getAllTasks(@NotNull String currentHost)
+    {
+        InstanceMetadata localInstance = instancesMetadata.instanceFromHost(currentHost);
+        if (localInstance == null)
+        {
+            throw new IllegalStateException("No instance found for host: " + currentHost);
+        }
+

Review Comment:
   removed



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationGetFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.livemigration.LiveMigrationTask;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler for retrieving a specific live migration files verification task by task ID.
+ * Returns the task details if found, or a 404 error if the task does not exist on the specified host.
+ */
+public class LiveMigrationGetFilesVerificationTaskHandler extends AbstractHandler<String> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationGetFilesVerificationTaskHandler.class);
+
+    private final FilesVerificationTaskManager taskManager;
+
+    /**
+     * Constructs a handler with the provided {@code metadataFetcher}
+     *
+     * @param metadataFetcher the interface to retrieve instance metadata
+     * @param executorPools   the executor pools for blocking executions
+     * @param validator       a validator instance to validate Cassandra-specific input
+     */
+    @Inject
+    public LiveMigrationGetFilesVerificationTaskHandler(InstanceMetadataFetcher metadataFetcher,
+                                                        ExecutorPools executorPools,
+                                                        CassandraInputValidator validator,
+                                                        FilesVerificationTaskManager taskManager)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.taskManager = taskManager;
+    }
+
+    @Override
+    protected String extractParamsOrThrow(RoutingContext context)
+    {
+        String taskId = context.pathParam("taskId");
+        if (taskId == null || taskId.isEmpty())
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST, "taskId is required");
+        }
+
+        return taskId;
+    }
+
+    @Override
+    protected void handleInternal(RoutingContext context,
+                                  HttpServerRequest httpRequest,
+                                  @NotNull String host,
+                                  SocketAddress remoteAddress,
+                                  String taskId)
+    {
+        try
+        {
+            LiveMigrationTask<LiveMigrationFilesVerificationResponse> task = taskManager.getTask(taskId, host);
+            LOGGER.info("Found live migration task with taskId={} on host={}", taskId, host);

Review Comment:
   done



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCreateFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.json.DecodeException;
+import io.vertx.core.json.Json;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationTaskCreationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationInvalidRequestException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.NoSuchCassandraInstanceException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.ACCEPTED;
+import static io.netty.handler.codec.http.HttpResponseStatus.BAD_REQUEST;
+import static io.netty.handler.codec.http.HttpResponseStatus.CONFLICT;
+import static io.netty.handler.codec.http.HttpResponseStatus.SERVICE_UNAVAILABLE;
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.common.ApiEndpointsV1.LIVE_MIGRATION_FILES_VERIFICATION_TASKS_ROUTE;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * HTTP handler for creating file digest verification tasks during live migration.
+ * Manages concurrent verification tasks per instance and orchestrates the verification process.
+ */
+public class LiveMigrationCreateFilesVerificationTaskHandler extends AbstractHandler<LiveMigrationFilesVerificationRequest> implements AccessProtected

Review Comment:
   done



##########
client-common/src/main/java/org/apache/cassandra/sidecar/common/response/LiveMigrationTaskCreationResponse.java:
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.common.response;
+
+import java.util.Objects;
+
+import com.fasterxml.jackson.annotation.JsonCreator;
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+/**
+ * Response object returned when a live migration task is created.
+ * Contains the task identifier and a URL to query the task status.
+ */
+public class LiveMigrationTaskCreationResponse
+{
+    private final String taskId;
+    private final String statusUrl;
+
+    @JsonCreator
+    public LiveMigrationTaskCreationResponse(@JsonProperty("taskId") String taskId,
+                                             @JsonProperty("statusUrl") String statusUrl)
+    {
+        Objects.requireNonNull(taskId, "taskId cannot be null");
+        Objects.requireNonNull(statusUrl, "statusUrl cannot be null");
+
+        this.taskId = taskId;
+        this.statusUrl = statusUrl;

Review Comment:
   done



##########
client-common/src/test/java/org/apache/cassandra/sidecar/common/response/LiveMigrationFilesVerificationResponseTest.java:
##########
@@ -0,0 +1,322 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.common.response;
+
+import org.junit.jupiter.api.Test;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+class LiveMigrationFilesVerificationResponseTest
+{
+    private final ObjectMapper mapper = new ObjectMapper();
+
+    @Test
+    void testSerializationRoundTrip() throws Exception

Review Comment:
   done



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCancelFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.livemigration.LiveMigrationTask;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler for canceling an active live migration files verification task.
+ * Accepts a task ID and cancels the corresponding verification task on the specified host.
+ */
+public class LiveMigrationCancelFilesVerificationTaskHandler extends AbstractHandler<String> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationCancelFilesVerificationTaskHandler.class);
+
+    private final FilesVerificationTaskManager taskManager;
+
+    /**
+     * Constructs a handler with the provided {@code metadataFetcher}
+     *
+     * @param metadataFetcher the interface to retrieve instance metadata
+     * @param executorPools   the executor pools for blocking executions
+     * @param validator       a validator instance to validate Cassandra-specific input
+     */
+    @Inject
+    public LiveMigrationCancelFilesVerificationTaskHandler(InstanceMetadataFetcher metadataFetcher,
+                                                           ExecutorPools executorPools,
+                                                           CassandraInputValidator validator,
+                                                           FilesVerificationTaskManager taskManager)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.taskManager = taskManager;
+    }
+
+    @Override
+    protected String extractParamsOrThrow(RoutingContext context)
+    {
+        return context.pathParam("taskId");

Review Comment:
   done



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationDigestHandlerWrapper.java:
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import com.google.inject.Inject;
+import io.vertx.core.Handler;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFileDigestRequest;
+
+import static org.apache.cassandra.sidecar.common.request.LiveMigrationFileDigestRequest.DIGEST_ALGORITHM_PARAM;
+
+/**
+ * Wrapper handler that conditionally delegates to {@link LiveMigrationFileDigestHandler} based on the presence
+ * of the {@link LiveMigrationFileDigestRequest#DIGEST_ALGORITHM_PARAM} query parameter, otherwise
+ * passes control to the next handler in the chain.
+ */
+public class LiveMigrationDigestHandlerWrapper implements Handler<RoutingContext>

Review Comment:
   Realised that I missed to mark multiple other handlers as Singleton. Marked other handlers too as Singleton.



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCreateFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.json.DecodeException;
+import io.vertx.core.json.Json;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationTaskCreationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationInvalidRequestException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.NoSuchCassandraInstanceException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.ACCEPTED;
+import static io.netty.handler.codec.http.HttpResponseStatus.BAD_REQUEST;
+import static io.netty.handler.codec.http.HttpResponseStatus.CONFLICT;
+import static io.netty.handler.codec.http.HttpResponseStatus.SERVICE_UNAVAILABLE;
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.common.ApiEndpointsV1.LIVE_MIGRATION_FILES_VERIFICATION_TASKS_ROUTE;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * HTTP handler for creating file digest verification tasks during live migration.
+ * Manages concurrent verification tasks per instance and orchestrates the verification process.
+ */
+public class LiveMigrationCreateFilesVerificationTaskHandler extends AbstractHandler<LiveMigrationFilesVerificationRequest> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationCreateFilesVerificationTaskHandler.class);
+
+    private final FilesVerificationTaskManager filesVerificationTaskManager;
+    private final LiveMigrationMap liveMigrationMap;
+
+    /**
+     * Constructs a handler with the provided {@code metadataFetcher}
+     *
+     * @param metadataFetcher the interface to retrieve instance metadata
+     * @param executorPools   the executor pools for blocking executions
+     * @param validator       a validator instance to validate Cassandra-specific input
+     */
+    @Inject
+    protected LiveMigrationCreateFilesVerificationTaskHandler(InstanceMetadataFetcher metadataFetcher,
+                                                              ExecutorPools executorPools,
+                                                              CassandraInputValidator validator,
+                                                              LiveMigrationMap liveMigrationMap,
+                                                              FilesVerificationTaskManager filesVerificationTaskManager)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.liveMigrationMap = liveMigrationMap;
+        this.filesVerificationTaskManager = filesVerificationTaskManager;
+    }
+
+    @Override
+    protected LiveMigrationFilesVerificationRequest extractParamsOrThrow(RoutingContext context)
+    {
+        try
+        {
+            return Json.decodeValue(context.body().buffer(), LiveMigrationFilesVerificationRequest.class);
+        }
+        catch (DecodeException decodeException)
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST,
+                                    "Failed to parse request body, please ensure that the request is valid.",
+                                    decodeException);
+        }
+        catch (IllegalArgumentException e)
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST, e.getMessage(), e);
+        }
+    }
+
+    @Override
+    protected void handleInternal(RoutingContext context,
+                                  HttpServerRequest httpRequest,
+                                  @NotNull String host,
+                                  SocketAddress remoteAddress,
+                                  LiveMigrationFilesVerificationRequest request)
+    {
+        LOGGER.debug("Received files verification request for host {} with maxConcurrency {}",

Review Comment:
   removed it



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationGetAllFilesVerificationTasksHandler.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import com.google.inject.Inject;
+import io.vertx.core.Handler;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.livemigration.LiveMigrationTask;
+
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.handlers.AbstractHandler.extractHostAddressWithoutPort;
+
+/**
+ * Handler for retrieving all active live migration files verification tasks.
+ * Returns a list of all verification tasks running on the local host.
+ */
+public class LiveMigrationGetAllFilesVerificationTasksHandler implements Handler<RoutingContext>, AccessProtected

Review Comment:
   Done



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationFileDigestHandler.java:
##########
@@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.Vertx;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.acl.authorization.BasicPermissions;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFileDigestRequest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.handlers.FileStreamHandler;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.DigestAlgorithm;
+import org.apache.cassandra.sidecar.utils.DigestAlgorithmFactory;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.INTERNAL_SERVER_ERROR;
+import static org.apache.cassandra.sidecar.common.request.LiveMigrationFileDigestRequest.DIGEST_ALGORITHM_PARAM;
+import static org.apache.cassandra.sidecar.utils.AsyncFileDigestCalculator.calculateDigest;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler for calculating and returning file digests during live migration.
+ * Supported digest algorithms are determined by {@link DigestAlgorithmFactory} and specified
+ * via the {@link LiveMigrationFileDigestRequest#DIGEST_ALGORITHM_PARAM} query parameter.
+ */
+public class LiveMigrationFileDigestHandler extends AbstractHandler<DigestAlgorithm> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationFileDigestHandler.class);
+
+    private final Vertx vertx;
+    private final DigestAlgorithmFactory digestAlgorithmFactory;
+
+    @Inject
+    public LiveMigrationFileDigestHandler(InstanceMetadataFetcher metadataFetcher,
+                                          ExecutorPools executorPools,
+                                          CassandraInputValidator validator,
+                                          Vertx vertx,
+                                          DigestAlgorithmFactory digestAlgorithmFactory)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.vertx = vertx;
+        this.digestAlgorithmFactory = digestAlgorithmFactory;
+    }
+
+    @Override
+    protected DigestAlgorithm extractParamsOrThrow(RoutingContext context)
+    {
+        String digestAlgorithmParam = getDigestAlgorithmParam(context);
+        try
+        {
+            return digestAlgorithmFactory.getDigestAlgorithm(digestAlgorithmParam, 0);
+        }
+        catch (IllegalArgumentException e)
+        {
+            LOGGER.error("Unexpected error while getting digest algorithm for {}", digestAlgorithmParam, e);
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST, e.getMessage());
+        }
+    }
+
+    @Override
+    protected void handleInternal(RoutingContext context,
+                                  HttpServerRequest httpRequest,
+                                  @NotNull String host,
+                                  SocketAddress remoteAddress,
+                                  DigestAlgorithm digestAlgorithm)
+    {
+        String file = context.get(FileStreamHandler.FILE_PATH_CONTEXT_KEY);
+
+        if (file == null)
+        {
+            LOGGER.error("File path not found in context");
+            context.fail(wrapHttpException(INTERNAL_SERVER_ERROR, "File path not available"));
+            return;
+        }
+        calculateDigest(vertx, file, digestAlgorithm)
+        .onComplete(ar -> {

Review Comment:
   > digestVerifier.verify(file)
   
   Is returning path and 'DigestVerifier' is validating path against known digest. So re-using `org.apache.cassandra.sidecar.utils.AsyncFileDigestCalculator#calculateDigest(io.vertx.core.Vertx, java.lang.String, org.apache.cassandra.sidecar.utils.DigestAlgorithm)` which is used by DigestVerifier too. 



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationTaskManager.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+
+import com.google.common.annotations.VisibleForTesting;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import org.apache.cassandra.sidecar.cluster.InstancesMetadata;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Centralized singleton manager for all live migration tasks across Cassandra instances.
+ * Enforces the constraint that only one {@link LiveMigrationTask} (of any type) can be active per instance at a time,
+ * preventing concurrent data copy and files verification operations that could lead to resource conflicts or data integrity issues.
+ * This singleton is shared across {@link DataCopyTaskManager} and {@link FilesVerificationTaskManager} to ensure
+ * proper coordination and mutual exclusion of tasks per instance.
+ */
+@Singleton
+public class LiveMigrationTaskManager
+{
+    @VisibleForTesting
+    final ConcurrentHashMap<Integer, LiveMigrationTask<?>> currentTasks = new ConcurrentHashMap<>();
+
+    private final InstancesMetadata instancesMetadata;
+
+    @Inject
+    public LiveMigrationTaskManager(InstancesMetadata instancesMetadata)
+    {
+        this.instancesMetadata = instancesMetadata;
+    }
+
+    /**
+     * Attempts to submit a new task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time.
+     *
+     * @param instanceId the instance ID
+     * @param newTask    the task to submit
+     * @return true if the task was accepted, false if another task is already in progress
+     */
+    public boolean submitTask(int instanceId, LiveMigrationTask<?> newTask)
+    {
+        return currentTasks.compute(instanceId, (integer, taskInMap) -> {

Review Comment:
   done



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCreateFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.json.DecodeException;
+import io.vertx.core.json.Json;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationTaskCreationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationInvalidRequestException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.NoSuchCassandraInstanceException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.ACCEPTED;
+import static io.netty.handler.codec.http.HttpResponseStatus.BAD_REQUEST;
+import static io.netty.handler.codec.http.HttpResponseStatus.CONFLICT;
+import static io.netty.handler.codec.http.HttpResponseStatus.SERVICE_UNAVAILABLE;
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.common.ApiEndpointsV1.LIVE_MIGRATION_FILES_VERIFICATION_TASKS_ROUTE;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * HTTP handler for creating file digest verification tasks during live migration.
+ * Manages concurrent verification tasks per instance and orchestrates the verification process.
+ */
+public class LiveMigrationCreateFilesVerificationTaskHandler extends AbstractHandler<LiveMigrationFilesVerificationRequest> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationCreateFilesVerificationTaskHandler.class);
+
+    private final FilesVerificationTaskManager filesVerificationTaskManager;
+    private final LiveMigrationMap liveMigrationMap;
+
+    /**
+     * Constructs a handler with the provided {@code metadataFetcher}
+     *
+     * @param metadataFetcher the interface to retrieve instance metadata
+     * @param executorPools   the executor pools for blocking executions
+     * @param validator       a validator instance to validate Cassandra-specific input
+     */
+    @Inject
+    protected LiveMigrationCreateFilesVerificationTaskHandler(InstanceMetadataFetcher metadataFetcher,
+                                                              ExecutorPools executorPools,
+                                                              CassandraInputValidator validator,
+                                                              LiveMigrationMap liveMigrationMap,
+                                                              FilesVerificationTaskManager filesVerificationTaskManager)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.liveMigrationMap = liveMigrationMap;
+        this.filesVerificationTaskManager = filesVerificationTaskManager;
+    }
+
+    @Override
+    protected LiveMigrationFilesVerificationRequest extractParamsOrThrow(RoutingContext context)
+    {
+        try
+        {
+            return Json.decodeValue(context.body().buffer(), LiveMigrationFilesVerificationRequest.class);
+        }
+        catch (DecodeException decodeException)
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST,
+                                    "Failed to parse request body, please ensure that the request is valid.",
+                                    decodeException);
+        }
+        catch (IllegalArgumentException e)
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST, e.getMessage(), e);
+        }
+    }
+
+    @Override
+    protected void handleInternal(RoutingContext context,
+                                  HttpServerRequest httpRequest,
+                                  @NotNull String host,
+                                  SocketAddress remoteAddress,
+                                  LiveMigrationFilesVerificationRequest request)
+    {
+        LOGGER.debug("Received files verification request for host {} with maxConcurrency {}",
+                     host, request.maxConcurrency());
+        InstanceMetadata localInstanceMetadata;
+        try
+        {
+            localInstanceMetadata = metadataFetcher.instance(host);
+        }
+        catch (NoSuchCassandraInstanceException e)
+        {
+            LOGGER.error("Failed to fetch instance metadata for host={}", host);
+            context.fail(wrapHttpException(SERVICE_UNAVAILABLE, e));
+            return;
+        }

Review Comment:
   Wanted to throw 503 instead of 500. So catching it and wrapping it into HttpException.



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationTaskManager.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+
+import com.google.common.annotations.VisibleForTesting;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import org.apache.cassandra.sidecar.cluster.InstancesMetadata;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Centralized singleton manager for all live migration tasks across Cassandra instances.
+ * Enforces the constraint that only one {@link LiveMigrationTask} (of any type) can be active per instance at a time,
+ * preventing concurrent data copy and files verification operations that could lead to resource conflicts or data integrity issues.
+ * This singleton is shared across {@link DataCopyTaskManager} and {@link FilesVerificationTaskManager} to ensure
+ * proper coordination and mutual exclusion of tasks per instance.
+ */
+@Singleton
+public class LiveMigrationTaskManager
+{
+    @VisibleForTesting
+    final ConcurrentHashMap<Integer, LiveMigrationTask<?>> currentTasks = new ConcurrentHashMap<>();
+
+    private final InstancesMetadata instancesMetadata;
+
+    @Inject
+    public LiveMigrationTaskManager(InstancesMetadata instancesMetadata)
+    {
+        this.instancesMetadata = instancesMetadata;
+    }
+
+    /**
+     * Attempts to submit a new task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time.
+     *
+     * @param instanceId the instance ID
+     * @param newTask    the task to submit
+     * @return true if the task was accepted, false if another task is already in progress
+     */
+    public boolean submitTask(int instanceId, LiveMigrationTask<?> newTask)
+    {
+        return currentTasks.compute(instanceId, (integer, taskInMap) -> {
+            if (taskInMap == null)
+            {
+                return newTask;
+            }
+
+            if (!taskInMap.isCompleted())
+            {
+                // Reject new task if existing task is still in progress
+                return taskInMap;
+            }
+            else
+            {
+                // Accept new task if existing task has completed
+                return newTask;
+            }
+        }) == newTask;
+    }
+
+    /**
+     * Returns all live migration tasks for given currentHost.
+     * This includes both active and completed tasks that haven't been replaced.
+     *
+     * @param currentHost the host where sidecar is running
+     * @return list containing at most one task (empty if no task has ever been submitted for this host)
+     */
+    @SuppressWarnings("ConstantValue")
+    public List<LiveMigrationTask<?>> getAllTasks(@NotNull String currentHost)
+    {
+        InstanceMetadata localInstance = instancesMetadata.instanceFromHost(currentHost);
+        if (localInstance == null)
+        {
+            throw new IllegalStateException("No instance found for host: " + currentHost);
+        }
+
+        LiveMigrationTask<?> task = currentTasks.get(localInstance.id());
+        return task == null ? Collections.emptyList() : Collections.singletonList(task);
+    }
+
+    /**
+     * Returns the live migration task with the specified task ID.
+     *
+     * @param taskId      ID of the task to retrieve
+     * @param currentHost the host where sidecar is running
+     * @return the LiveMigrationTask matching the given taskId
+     * @throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException if no task found with the given ID
+     */
+    public LiveMigrationTask<?> getTask(@NotNull String taskId,
+                                        @NotNull String currentHost) throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException
+    {
+        return getLiveMigrationTask(taskId, currentHost);
+    }
+
+    /**
+     * Cancels the live migration task with the specified task ID.
+     *
+     * @param taskId      ID of the task to cancel
+     * @param currentHost the host where sidecar is running
+     * @return the cancelled LiveMigrationTask
+     * @throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException if no task found with the given ID
+     */
+    public LiveMigrationTask<?> cancelTask(@NotNull String taskId,
+                                           @NotNull String currentHost) throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException
+    {
+        LiveMigrationTask<?> taskInProgress = getLiveMigrationTask(taskId, currentHost);
+
+        // Cancelling the task
+        taskInProgress.cancel();
+
+        return taskInProgress;
+    }
+
+    @SuppressWarnings("ConstantValue")
+    private LiveMigrationTask<?> getLiveMigrationTask(@NotNull String taskId, @NotNull String currentHost)
+    {
+        InstanceMetadata localInstance = instancesMetadata.instanceFromHost(currentHost);
+        if (localInstance == null)
+        {
+            throw new IllegalStateException("No instance found for host: " + currentHost);
+        }

Review Comment:
   removed



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationGetFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.livemigration.LiveMigrationTask;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * Handler for retrieving a specific live migration files verification task by task ID.
+ * Returns the task details if found, or a 404 error if the task does not exist on the specified host.
+ */
+public class LiveMigrationGetFilesVerificationTaskHandler extends AbstractHandler<String> implements AccessProtected

Review Comment:
   done



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationFilesVerificationTaskFactory.java:
##########
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import io.vertx.core.Vertx;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+import org.apache.cassandra.sidecar.utils.DigestVerifierFactory;
+import org.apache.cassandra.sidecar.utils.SidecarClientProvider;
+
+/**
+ * Factory for creating {@link LiveMigrationFilesVerificationTask} instances used to verify file digests
+ * between source and destination instances during live migration operation.
+ */
+@Singleton
+public class LiveMigrationFilesVerificationTaskFactory
+{
+    private final Vertx vertx;
+    private final SidecarClientProvider sidecarClientProvider;
+    private final ExecutorPools executorPools;
+    private final DigestVerifierFactory digestVerifierFactory;
+    private final SidecarConfiguration sidecarConfiguration;
+
+    @Inject
+    public LiveMigrationFilesVerificationTaskFactory(Vertx vertx,
+                                                     ExecutorPools executorPools,
+                                                     SidecarConfiguration sidecarConfiguration,
+                                                     SidecarClientProvider sidecarClientProvider,
+                                                     DigestVerifierFactory digestVerifierFactory)
+    {
+        this.vertx = vertx;
+        this.sidecarClientProvider = sidecarClientProvider;
+        this.executorPools = executorPools;
+        this.digestVerifierFactory = digestVerifierFactory;
+        this.sidecarConfiguration = sidecarConfiguration;
+    }
+
+    public LiveMigrationTask<LiveMigrationFilesVerificationResponse> create(String id,
+                                                                            String source,
+                                                                            int port,
+                                                                            LiveMigrationFilesVerificationRequest request,
+                                                                            InstanceMetadata localInstanceMetadata)
+    {
+        return LiveMigrationFilesVerificationTask.builder()
+                                                 .id(id)
+                                                 .source(source)
+                                                 .port(port)
+                                                 .vertx(vertx)
+                                                 .executorPools(executorPools)
+                                                 .sidecarClient(sidecarClientProvider.get())

Review Comment:
   SidecarClientProvider is Singleton and `.get()` is returning its internal initialized client. Only one client instance is used.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSSIDECAR-226 Adding endpoint for verifying files post data copy during live migration [cassandra-sidecar]

Posted by "yifan-c (via GitHub)" <gi...@apache.org>.
yifan-c commented on code in PR #309:
URL: https://github.com/apache/cassandra-sidecar/pull/309#discussion_r2887687485


##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationFilesVerificationTask.java:
##########
@@ -0,0 +1,643 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.Callable;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.vertx.core.Future;
+import io.vertx.core.MultiMap;
+import io.vertx.core.Promise;
+import io.vertx.core.Vertx;
+import org.apache.cassandra.sidecar.client.SidecarClient;
+import org.apache.cassandra.sidecar.client.SidecarInstanceImpl;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.DataObjectBuilder;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.request.data.Digest;
+import org.apache.cassandra.sidecar.common.request.data.MD5Digest;
+import org.apache.cassandra.sidecar.common.request.data.XXHash32Digest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.common.response.InstanceFileInfo;
+import org.apache.cassandra.sidecar.common.response.InstanceFilesListResponse;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.AsyncConcurrentTaskExecutor;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.DigestMismatchException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.FileVerificationFailureException;
+import org.apache.cassandra.sidecar.utils.DigestVerifierFactory;
+import org.jetbrains.annotations.NotNull;
+import org.jetbrains.annotations.VisibleForTesting;
+
+import static org.apache.cassandra.sidecar.common.response.InstanceFileInfo.FileType.DIRECTORY;
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationInstanceMetadataUtil.localPath;
+
+/**
+ * Verifies file integrity between source and destination instances during live migration
+ * by comparing file lists and validating file digests using configurable digest algorithms.
+ *
+ * <p>The verification process consists of three stages:
+ * <ol>
+ *   <li>Fetch file lists from both source and destination instances concurrently</li>
+ *   <li>Compare file metadata (size, type, modification time) - fails fast on mismatches</li>
+ *   <li>Verify cryptographic digests (MD5 or XXHash32) with configurable concurrency</li>
+ * </ol>
+ *
+ * <p>The task supports cancellation, and tracks detailed metrics including
+ * metadata matches/mismatches, digest verification results, and failure counts.
+ */
+public class LiveMigrationFilesVerificationTask implements LiveMigrationTask<LiveMigrationFilesVerificationResponse>
+{
+    public static final String FILES_VERIFICATION_TASK_TYPE = "files-verification-task";
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationFilesVerificationTask.class);
+    private final Vertx vertx;
+    private final String id;
+    private final String source;
+    private final int port;
+    // Instance metadata of current instance
+    private final InstanceMetadata instanceMetadata;
+    private final ExecutorPools executorPools;
+    private final LiveMigrationFilesVerificationRequest request;
+    private final SidecarClient sidecarClient;
+    private final LiveMigrationConfiguration liveMigrationConfiguration;
+    private final DigestVerifierFactory digestVerifierFactory;
+    private final AtomicReference<State> state;
+    private final Promise<Void> completionPromise = Promise.promise();
+    private final AtomicInteger filesNotFoundAtSource = new AtomicInteger(0);
+    private final AtomicInteger filesNotFoundAtDestination = new AtomicInteger(0);
+    private final AtomicInteger metadataMatched = new AtomicInteger(0);
+    private final AtomicInteger metadataMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestVerificationFailures = new AtomicInteger(0);
+    private final AtomicInteger filesMatched = new AtomicInteger(0);
+    private final String logPrefix;
+    private final AtomicBoolean cancelled = new AtomicBoolean(false);
+    private volatile AsyncConcurrentTaskExecutor<String> asyncConcurrentTaskExecutor;
+
+    @VisibleForTesting
+    protected LiveMigrationFilesVerificationTask(Builder builder)
+    {
+        this.vertx = builder.vertx;
+        this.id = builder.id;
+        this.source = builder.source;
+        this.port = builder.port;
+        this.instanceMetadata = builder.instanceMetadata;
+        this.executorPools = builder.executorPools;
+        this.request = builder.request;
+        this.sidecarClient = builder.sidecarClient;
+        this.liveMigrationConfiguration = builder.liveMigrationConfiguration;
+        this.digestVerifierFactory = builder.digestVerifierFactory;
+        this.logPrefix = "fileVerifyTaskId=" + id;
+        this.state = new AtomicReference<>(State.NOT_STARTED);
+    }
+
+    public static Builder builder()
+    {
+        return new Builder();
+    }
+
+    public Future<Void> validate()
+    {
+        return abortIfCancelled(new Object())
+               // Get files info from both source and destination
+               .compose(v -> Future.join(listFilesInLocal(), fetchSourceFileList()))
+
+               // Compare the files information before calculating digests
+               .compose(this::abortIfCancelled)
+               .compose(cf -> compareFilesMeta(cf.resultAt(0), cf.resultAt(1)))
+
+               // Create digest comparison tasks
+               .compose(this::abortIfCancelled)
+               .compose(this::getDigestComparisonTasks)
+
+               // Compare digests
+               .compose(this::abortIfCancelled)
+               .compose(this::compareDigests)
+
+               .andThen(ar -> {
+                   if (ar.failed())
+                   {
+                       LOGGER.error("{} Files verification task failed during validation", logPrefix, ar.cause());
+                       // Fail the task if it is in IN_PROGRESS state.
+                       state.compareAndSet(State.IN_PROGRESS, State.FAILED);
+                   }
+                   else
+                   {
+                       State currentState = state.compareAndExchange(State.IN_PROGRESS, State.COMPLETED);
+                       if (currentState == State.CANCELLED)
+                       {
+                           LOGGER.debug("{} Task was cancelled before completion", logPrefix);
+                       }
+                       else if (currentState != State.IN_PROGRESS)
+                       {
+                           LOGGER.warn("{} Unexpected state transition failure. State was {}", logPrefix, currentState);
+                       }
+                       else
+                       {
+                           LOGGER.info("{} Files verification task completed successfully", logPrefix);
+                       }
+                   }
+               });
+    }
+
+    private Future<List<InstanceFileInfo>> listFilesInLocal()
+    {
+        return executorPools.internal()
+                            .executeBlocking(() -> new CassandraInstanceFilesImpl(instanceMetadata,
+                                                                                  liveMigrationConfiguration)
+                                                   .files());
+    }
+
+    private Future<InstanceFilesListResponse> fetchSourceFileList()
+    {
+        return Future.fromCompletionStage(
+                     sidecarClient.liveMigrationListInstanceFilesAsync(new SidecarInstanceImpl(source, port)))
+                     .onFailure(cause -> LOGGER.error("{} Failed to obtain list of files from source {}:{}",
+                                                      logPrefix, source, port, cause));
+    }
+
+    private @NotNull Future<List<InstanceFileInfo>> compareFilesMeta(List<InstanceFileInfo> localFiles,
+                                                                     InstanceFilesListResponse sourceFiles)
+    {
+        Map<String, InstanceFileInfo> filesAtLocal =
+        localFiles.stream()
+                  .collect(Collectors.toMap(fileInfo -> fileInfo.fileUrl, fileInfo -> fileInfo));
+
+        Map<String, InstanceFileInfo> filesAtSource =
+        sourceFiles.getFiles().stream()
+                   .collect(Collectors.toMap(fileInfo -> fileInfo.fileUrl, fileInfo -> fileInfo));
+
+        for (Map.Entry<String, InstanceFileInfo> localFileEntry : filesAtLocal.entrySet())
+        {
+            String fileUrl = localFileEntry.getKey();
+            InstanceFileInfo localFile = localFileEntry.getValue();
+            if (filesAtSource.containsKey(fileUrl))
+            {
+                InstanceFileInfo sourceFile = filesAtSource.get(fileUrl);
+                // compare files metadata
+                if (localFile.equals(sourceFile) ||
+                    (localFile.fileType == DIRECTORY && sourceFile.fileType == DIRECTORY))
+                {
+                    // File info matches if either of them are directories or
+                    // their size, last modified time etc... matches
+                    LOGGER.debug("{} {} file metadata matched with source", logPrefix, fileUrl);
+                    metadataMatched.incrementAndGet();
+                }
+                else
+                {
+                    // Files metadata did not match
+                    metadataMismatches.incrementAndGet();
+                    LOGGER.error("{} {} did not match with source. Local file info={}, source file info={}",
+                                 logPrefix, fileUrl, localFile, sourceFile);
+                }
+
+                // done with processing current local file, remove it
+                filesAtSource.remove(fileUrl);
+            }
+            else
+            {
+                // File is not available at source, report it
+                LOGGER.error("{} File {} exists at destination but not found at source", logPrefix, fileUrl);
+                filesNotFoundAtSource.incrementAndGet();
+            }
+        }
+
+        for (Map.Entry<String, InstanceFileInfo> sourceFileEntry : filesAtSource.entrySet())
+        {
+            // Remaining files are not present at destination, report error for them
+            LOGGER.error("{} File {} exists at source but not found at destination", logPrefix, sourceFileEntry.getKey());
+            filesNotFoundAtDestination.incrementAndGet();
+        }
+
+        if (filesNotFoundAtDestination.get() > 0
+            || filesNotFoundAtSource.get() > 0
+            || metadataMismatches.get() > 0)
+        {
+            FileVerificationFailureException exception =
+            new FileVerificationFailureException("Files list did not match between source and destination");
+
+            return Future.failedFuture(exception);
+        }
+
+        return Future.succeededFuture(localFiles);
+    }
+
+    private @NotNull Future<List<Future<String>>> getDigestComparisonTasks(List<InstanceFileInfo> localFiles)
+    {
+        // now verify digests of each local file with source file
+        List<Callable<Future<String>>> tasks = new ArrayList<>(localFiles.size());
+        localFiles.stream()
+                  .filter(fileInfo -> fileInfo.fileType != DIRECTORY)
+                  .forEach(fileInfo -> tasks.add(() -> verifyDigest(fileInfo)));
+        asyncConcurrentTaskExecutor = new AsyncConcurrentTaskExecutor<>(vertx, tasks, request.maxConcurrency());
+        List<Future<String>> verifyTaskResults = asyncConcurrentTaskExecutor.start();
+        return Future.join(verifyTaskResults)
+                     .transform(ar -> Future.succeededFuture(verifyTaskResults));
+    }
+
+    private Future<Void> compareDigests(List<Future<String>> futures)
+    {
+        boolean verificationFailed = false;
+        for (Future<String> future : futures)
+        {
+            if (future.failed())
+            {
+                verificationFailed = true;
+                Throwable cause = future.cause();
+                if (cause instanceof DigestMismatchException)
+                {
+                    DigestMismatchException ex = (DigestMismatchException) cause;
+                    LOGGER.error("{} File digests did not match for {}.", logPrefix, ex.fileUrl(), cause);
+                    digestMismatches.incrementAndGet();
+                }
+                else
+                {
+                    LOGGER.error("{} Failed to verify digest for file. Check previous logs for file details",
+                                 logPrefix, cause);
+                    digestVerificationFailures.incrementAndGet();
+                }
+                continue;
+            }
+
+            // Digest verification succeeded
+            filesMatched.incrementAndGet();
+        }
+
+        if (verificationFailed)
+        {
+            return Future.failedFuture(
+            new FileVerificationFailureException("File digests did not match between source and destination"));
+        }
+        else
+        {
+            return Future.succeededFuture();
+        }
+    }
+
+    @VisibleForTesting
+    Future<String> verifyDigest(InstanceFileInfo fileInfo)
+    {
+        return getSourceFileDigest(fileInfo)
+               .compose(digest -> {
+                   String path = localPath(fileInfo.fileUrl, instanceMetadata).toAbsolutePath().toString();
+                   return digestVerifierFactory.verifier(MultiMap.caseInsensitiveMultiMap().addAll(digest.headers()))
+                                               .verify(path)
+                                               .compose(verified -> Future.succeededFuture(path))
+                                               .recover(cause -> Future.failedFuture(
+                                               new DigestMismatchException(path, fileInfo.fileUrl, cause)));
+               })
+               .onSuccess(filePath -> LOGGER.debug("{} Verified file {}", logPrefix, fileInfo.fileUrl))
+               .onFailure(cause -> LOGGER.error("{} Failed to verify file {}", logPrefix, fileInfo.fileUrl, cause));
+    }
+
+    private Future<Digest> getSourceFileDigest(InstanceFileInfo fileInfo)
+    {
+        return Future.fromCompletionStage(sidecarClient.liveMigrationFileDigestAsync(new SidecarInstanceImpl(source, port),
+                                                                                     fileInfo.fileUrl,
+                                                                                     request.digestAlgorithm()))
+                     .compose(this::toDigest);
+    }

Review Comment:
   5xx status code are considered server side error (unrelated to client activities). 
   4xx status codes are errors triggered by client. 
   IMO, 429 makes sense in this scenario. The error happens **only** when there are enough concurrent requests are issued by client. 
   The default retry policy does not react on 429 status code and no retries are performed. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSSIDECAR-226 Adding endpoint for verifying files post data copy during live migration [cassandra-sidecar]

Posted by "nvharikrishna (via GitHub)" <gi...@apache.org>.
nvharikrishna commented on code in PR #309:
URL: https://github.com/apache/cassandra-sidecar/pull/309#discussion_r3214707075


##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationTaskManager.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+
+import com.google.common.annotations.VisibleForTesting;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import org.apache.cassandra.sidecar.cluster.InstancesMetadata;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Centralized singleton manager for all live migration tasks across Cassandra instances.
+ * Enforces the constraint that only one {@link LiveMigrationTask} (of any type) can be active per instance at a time,
+ * preventing concurrent data copy and files verification operations that could lead to resource conflicts or data integrity issues.
+ * This singleton is shared across {@link DataCopyTaskManager} and {@link FilesVerificationTaskManager} to ensure
+ * proper coordination and mutual exclusion of tasks per instance.
+ */
+@Singleton
+public class LiveMigrationTaskManager
+{
+    @VisibleForTesting
+    final ConcurrentHashMap<Integer, LiveMigrationTask<?>> currentTasks = new ConcurrentHashMap<>();
+
+    private final InstancesMetadata instancesMetadata;
+
+    @Inject
+    public LiveMigrationTaskManager(InstancesMetadata instancesMetadata)
+    {
+        this.instancesMetadata = instancesMetadata;
+    }
+
+    /**
+     * Attempts to submit a new task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time.
+     *
+     * @param instanceId the instance ID
+     * @param newTask    the task to submit
+     * @return true if the task was accepted, false if another task is already in progress
+     */
+    public boolean submitTask(int instanceId, LiveMigrationTask<?> newTask)
+    {
+        return currentTasks.compute(instanceId, (integer, taskInMap) -> {
+            if (taskInMap == null)
+            {
+                return newTask;
+            }
+
+            if (!taskInMap.isCompleted())
+            {
+                // Reject new task if existing task is still in progress
+                return taskInMap;
+            }
+            else
+            {
+                // Accept new task if existing task has completed
+                return newTask;
+            }
+        }) == newTask;
+    }
+
+    /**
+     * Returns all live migration tasks for given currentHost.
+     * This includes both active and completed tasks that haven't been replaced.
+     *
+     * @param currentHost the host where sidecar is running
+     * @return list containing at most one task (empty if no task has ever been submitted for this host)
+     */
+    @SuppressWarnings("ConstantValue")
+    public List<LiveMigrationTask<?>> getAllTasks(@NotNull String currentHost)
+    {
+        InstanceMetadata localInstance = instancesMetadata.instanceFromHost(currentHost);
+        if (localInstance == null)
+        {
+            throw new IllegalStateException("No instance found for host: " + currentHost);
+        }
+
+        LiveMigrationTask<?> task = currentTasks.get(localInstance.id());
+        return task == null ? Collections.emptyList() : Collections.singletonList(task);
+    }
+
+    /**
+     * Returns the live migration task with the specified task ID.
+     *
+     * @param taskId      ID of the task to retrieve
+     * @param currentHost the host where sidecar is running
+     * @return the LiveMigrationTask matching the given taskId
+     * @throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException if no task found with the given ID
+     */
+    public LiveMigrationTask<?> getTask(@NotNull String taskId,
+                                        @NotNull String currentHost) throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException
+    {
+        return getLiveMigrationTask(taskId, currentHost);
+    }
+
+    /**
+     * Cancels the live migration task with the specified task ID.
+     *
+     * @param taskId      ID of the task to cancel
+     * @param currentHost the host where sidecar is running
+     * @return the cancelled LiveMigrationTask
+     * @throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException if no task found with the given ID
+     */
+    public LiveMigrationTask<?> cancelTask(@NotNull String taskId,
+                                           @NotNull String currentHost) throws LiveMigrationExceptions.LiveMigrationTaskNotFoundException
+    {
+        LiveMigrationTask<?> taskInProgress = getLiveMigrationTask(taskId, currentHost);
+
+        // Cancelling the task
+        taskInProgress.cancel();
+
+        return taskInProgress;
+    }
+
+    @SuppressWarnings("ConstantValue")
+    private LiveMigrationTask<?> getLiveMigrationTask(@NotNull String taskId, @NotNull String currentHost)
+    {
+        InstanceMetadata localInstance = instancesMetadata.instanceFromHost(currentHost);
+        if (localInstance == null)
+        {
+            throw new IllegalStateException("No instance found for host: " + currentHost);
+        }

Review Comment:
   Somehow missed to push this change, pushed it now.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSSIDECAR-226 Adding endpoint for verifying files post data copy during live migration [cassandra-sidecar]

Posted by "nvharikrishna (via GitHub)" <gi...@apache.org>.
nvharikrishna commented on code in PR #309:
URL: https://github.com/apache/cassandra-sidecar/pull/309#discussion_r2875781019


##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCreateFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.json.DecodeException;
+import io.vertx.core.json.Json;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationTaskCreationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.NoSuchCassandraInstanceException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.DigestAlgorithmFactory;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.ACCEPTED;
+import static io.netty.handler.codec.http.HttpResponseStatus.FORBIDDEN;
+import static io.netty.handler.codec.http.HttpResponseStatus.SERVICE_UNAVAILABLE;
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.common.ApiEndpointsV1.LIVE_MIGRATION_FILES_VERIFICATION_TASKS_ROUTE;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * HTTP handler for creating file digest verification tasks during live migration.
+ * Manages concurrent verification tasks per instance and orchestrates the verification process.
+ */
+public class LiveMigrationCreateFilesVerificationTaskHandler extends AbstractHandler<LiveMigrationFilesVerificationRequest> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationCreateFilesVerificationTaskHandler.class);
+
+    private final FilesVerificationTaskManager filesVerificationTaskManager;
+    private final LiveMigrationMap liveMigrationMap;
+    private final LiveMigrationConfiguration liveMigrationConfiguration;
+
+    /**
+     * Constructs a handler with the provided {@code metadataFetcher}
+     *
+     * @param metadataFetcher the interface to retrieve instance metadata
+     * @param executorPools   the executor pools for blocking executions
+     * @param validator       a validator instance to validate Cassandra-specific input
+     */
+    @Inject
+    protected LiveMigrationCreateFilesVerificationTaskHandler(InstanceMetadataFetcher metadataFetcher,
+                                                              ExecutorPools executorPools,
+                                                              CassandraInputValidator validator,
+                                                              LiveMigrationMap liveMigrationMap,
+                                                              FilesVerificationTaskManager filesVerificationTaskManager,
+                                                              SidecarConfiguration sidecarConfiguration)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.liveMigrationMap = liveMigrationMap;
+        this.filesVerificationTaskManager = filesVerificationTaskManager;
+        this.liveMigrationConfiguration = sidecarConfiguration.liveMigrationConfiguration();
+    }
+
+    @Override
+    protected LiveMigrationFilesVerificationRequest extractParamsOrThrow(RoutingContext context)
+    {
+        try
+        {
+            LiveMigrationFilesVerificationRequest request =
+            Json.decodeValue(context.body().buffer(), LiveMigrationFilesVerificationRequest.class);
+
+            if (request.maxConcurrency() > liveMigrationConfiguration.maxConcurrentFileRequests())
+            {
+                throw new IllegalArgumentException("Invalid maxConcurrency " + request.maxConcurrency() +
+                                                   ". It cannot be greater than " +
+                                                   liveMigrationConfiguration.maxConcurrentFileRequests());
+            }

Review Comment:
   De-duplicated the checks



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCreateFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.json.DecodeException;
+import io.vertx.core.json.Json;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationTaskCreationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.NoSuchCassandraInstanceException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.DigestAlgorithmFactory;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.ACCEPTED;
+import static io.netty.handler.codec.http.HttpResponseStatus.FORBIDDEN;
+import static io.netty.handler.codec.http.HttpResponseStatus.SERVICE_UNAVAILABLE;
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.common.ApiEndpointsV1.LIVE_MIGRATION_FILES_VERIFICATION_TASKS_ROUTE;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * HTTP handler for creating file digest verification tasks during live migration.
+ * Manages concurrent verification tasks per instance and orchestrates the verification process.
+ */
+public class LiveMigrationCreateFilesVerificationTaskHandler extends AbstractHandler<LiveMigrationFilesVerificationRequest> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationCreateFilesVerificationTaskHandler.class);
+
+    private final FilesVerificationTaskManager filesVerificationTaskManager;
+    private final LiveMigrationMap liveMigrationMap;
+    private final LiveMigrationConfiguration liveMigrationConfiguration;
+
+    /**
+     * Constructs a handler with the provided {@code metadataFetcher}
+     *
+     * @param metadataFetcher the interface to retrieve instance metadata
+     * @param executorPools   the executor pools for blocking executions
+     * @param validator       a validator instance to validate Cassandra-specific input
+     */
+    @Inject
+    protected LiveMigrationCreateFilesVerificationTaskHandler(InstanceMetadataFetcher metadataFetcher,
+                                                              ExecutorPools executorPools,
+                                                              CassandraInputValidator validator,
+                                                              LiveMigrationMap liveMigrationMap,
+                                                              FilesVerificationTaskManager filesVerificationTaskManager,
+                                                              SidecarConfiguration sidecarConfiguration)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.liveMigrationMap = liveMigrationMap;
+        this.filesVerificationTaskManager = filesVerificationTaskManager;
+        this.liveMigrationConfiguration = sidecarConfiguration.liveMigrationConfiguration();
+    }
+
+    @Override
+    protected LiveMigrationFilesVerificationRequest extractParamsOrThrow(RoutingContext context)
+    {
+        try
+        {
+            LiveMigrationFilesVerificationRequest request =
+            Json.decodeValue(context.body().buffer(), LiveMigrationFilesVerificationRequest.class);
+
+            if (request.maxConcurrency() > liveMigrationConfiguration.maxConcurrentFileRequests())
+            {
+                throw new IllegalArgumentException("Invalid maxConcurrency " + request.maxConcurrency() +
+                                                   ". It cannot be greater than " +
+                                                   liveMigrationConfiguration.maxConcurrentFileRequests());
+            }
+
+            // Validate digest algorithm without creating an instance
+            DigestAlgorithmFactory.validateAlgorithmName(request.digestAlgorithm());
+
+            return request;
+        }
+        catch (DecodeException decodeException)
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST,
+                                    "Failed to parse request body, please ensure that the request is valid.",
+                                    decodeException);
+        }
+        catch (IllegalArgumentException e)
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST, e.getMessage(), e);
+        }
+    }
+
+    @Override
+    protected void handleInternal(RoutingContext context,
+                                  HttpServerRequest httpRequest,
+                                  @NotNull String host,
+                                  SocketAddress remoteAddress,
+                                  LiveMigrationFilesVerificationRequest request)
+    {
+        LOGGER.debug("Received files verification request for host {} with maxConcurrency {}",
+                     host, request.maxConcurrency());
+        InstanceMetadata localInstanceMetadata;
+        try
+        {
+            localInstanceMetadata = metadataFetcher.instance(host);
+        }
+        catch (NoSuchCassandraInstanceException e)
+        {
+            LOGGER.error("Failed to fetch instance metadata for host={}", host);
+            context.fail(wrapHttpException(SERVICE_UNAVAILABLE, e));
+            return;
+        }
+
+        liveMigrationMap.getSource(host)
+                        .compose(source -> filesVerificationTaskManager.createTask(request, source, localInstanceMetadata))
+
+                        .onSuccess(task -> {
+                            LOGGER.info("Created files verification task {} for host {}", task.id(), host);
+                            context.response().setStatusCode(ACCEPTED.code());
+                            String statusUrl = LIVE_MIGRATION_FILES_VERIFICATION_TASKS_ROUTE + "/" + task.id();
+                            context.json(new LiveMigrationTaskCreationResponse(task.id(), statusUrl));
+                        })
+
+                        .onFailure(throwable -> {
+                            if (throwable instanceof LiveMigrationTaskInProgressException)
+                            {
+                                LOGGER.error("Cannot start a new files verification task for host {} " +
+                                             "while another live migration task is in progress.", host);
+                                context.fail(wrapHttpException(FORBIDDEN, throwable.getMessage(), throwable));

Review Comment:
   CONFLICT makes more sense. Updated it.



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationFilesVerificationTask.java:
##########
@@ -0,0 +1,643 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.Callable;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.vertx.core.Future;
+import io.vertx.core.MultiMap;
+import io.vertx.core.Promise;
+import io.vertx.core.Vertx;
+import org.apache.cassandra.sidecar.client.SidecarClient;
+import org.apache.cassandra.sidecar.client.SidecarInstanceImpl;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.DataObjectBuilder;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.request.data.Digest;
+import org.apache.cassandra.sidecar.common.request.data.MD5Digest;
+import org.apache.cassandra.sidecar.common.request.data.XXHash32Digest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.common.response.InstanceFileInfo;
+import org.apache.cassandra.sidecar.common.response.InstanceFilesListResponse;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.AsyncConcurrentTaskExecutor;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.DigestMismatchException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.FileVerificationFailureException;
+import org.apache.cassandra.sidecar.utils.DigestVerifierFactory;
+import org.jetbrains.annotations.NotNull;
+import org.jetbrains.annotations.VisibleForTesting;
+
+import static org.apache.cassandra.sidecar.common.response.InstanceFileInfo.FileType.DIRECTORY;
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationInstanceMetadataUtil.localPath;
+
+/**
+ * Verifies file integrity between source and destination instances during live migration
+ * by comparing file lists and validating file digests using configurable digest algorithms.
+ *
+ * <p>The verification process consists of three stages:
+ * <ol>
+ *   <li>Fetch file lists from both source and destination instances concurrently</li>
+ *   <li>Compare file metadata (size, type, modification time) - fails fast on mismatches</li>

Review Comment:
   I intended to say that the task won’t proceed to verifying file digests and will stop here if there are metadata mismatches. However, I understand your point. I’ve updated the text to avoid any confusion.



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationFilesVerificationTask.java:
##########
@@ -0,0 +1,643 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.Callable;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.vertx.core.Future;
+import io.vertx.core.MultiMap;
+import io.vertx.core.Promise;
+import io.vertx.core.Vertx;
+import org.apache.cassandra.sidecar.client.SidecarClient;
+import org.apache.cassandra.sidecar.client.SidecarInstanceImpl;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.DataObjectBuilder;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.request.data.Digest;
+import org.apache.cassandra.sidecar.common.request.data.MD5Digest;
+import org.apache.cassandra.sidecar.common.request.data.XXHash32Digest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.common.response.InstanceFileInfo;
+import org.apache.cassandra.sidecar.common.response.InstanceFilesListResponse;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.AsyncConcurrentTaskExecutor;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.DigestMismatchException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.FileVerificationFailureException;
+import org.apache.cassandra.sidecar.utils.DigestVerifierFactory;
+import org.jetbrains.annotations.NotNull;
+import org.jetbrains.annotations.VisibleForTesting;
+
+import static org.apache.cassandra.sidecar.common.response.InstanceFileInfo.FileType.DIRECTORY;
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationInstanceMetadataUtil.localPath;
+
+/**
+ * Verifies file integrity between source and destination instances during live migration
+ * by comparing file lists and validating file digests using configurable digest algorithms.
+ *
+ * <p>The verification process consists of three stages:
+ * <ol>
+ *   <li>Fetch file lists from both source and destination instances concurrently</li>
+ *   <li>Compare file metadata (size, type, modification time) - fails fast on mismatches</li>
+ *   <li>Verify cryptographic digests (MD5 or XXHash32) with configurable concurrency</li>
+ * </ol>
+ *
+ * <p>The task supports cancellation, and tracks detailed metrics including
+ * metadata matches/mismatches, digest verification results, and failure counts.
+ */
+public class LiveMigrationFilesVerificationTask implements LiveMigrationTask<LiveMigrationFilesVerificationResponse>
+{
+    public static final String FILES_VERIFICATION_TASK_TYPE = "files-verification-task";
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationFilesVerificationTask.class);
+    private final Vertx vertx;
+    private final String id;
+    private final String source;
+    private final int port;
+    // Instance metadata of current instance
+    private final InstanceMetadata instanceMetadata;
+    private final ExecutorPools executorPools;
+    private final LiveMigrationFilesVerificationRequest request;
+    private final SidecarClient sidecarClient;
+    private final LiveMigrationConfiguration liveMigrationConfiguration;
+    private final DigestVerifierFactory digestVerifierFactory;
+    private final AtomicReference<State> state;
+    private final Promise<Void> completionPromise = Promise.promise();
+    private final AtomicInteger filesNotFoundAtSource = new AtomicInteger(0);
+    private final AtomicInteger filesNotFoundAtDestination = new AtomicInteger(0);
+    private final AtomicInteger metadataMatched = new AtomicInteger(0);
+    private final AtomicInteger metadataMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestVerificationFailures = new AtomicInteger(0);
+    private final AtomicInteger filesMatched = new AtomicInteger(0);
+    private final String logPrefix;
+    private final AtomicBoolean cancelled = new AtomicBoolean(false);
+    private volatile AsyncConcurrentTaskExecutor<String> asyncConcurrentTaskExecutor;
+
+    @VisibleForTesting
+    protected LiveMigrationFilesVerificationTask(Builder builder)
+    {
+        this.vertx = builder.vertx;
+        this.id = builder.id;
+        this.source = builder.source;
+        this.port = builder.port;
+        this.instanceMetadata = builder.instanceMetadata;
+        this.executorPools = builder.executorPools;
+        this.request = builder.request;
+        this.sidecarClient = builder.sidecarClient;
+        this.liveMigrationConfiguration = builder.liveMigrationConfiguration;
+        this.digestVerifierFactory = builder.digestVerifierFactory;
+        this.logPrefix = "fileVerifyTaskId=" + id;
+        this.state = new AtomicReference<>(State.NOT_STARTED);
+    }
+
+    public static Builder builder()
+    {
+        return new Builder();
+    }
+
+    public Future<Void> validate()
+    {
+        return abortIfCancelled(new Object())
+               // Get files info from both source and destination
+               .compose(v -> Future.join(listFilesInLocal(), fetchSourceFileList()))
+
+               // Compare the files information before calculating digests
+               .compose(this::abortIfCancelled)
+               .compose(cf -> compareFilesMeta(cf.resultAt(0), cf.resultAt(1)))
+
+               // Create digest comparison tasks
+               .compose(this::abortIfCancelled)
+               .compose(this::getDigestComparisonTasks)
+
+               // Compare digests
+               .compose(this::abortIfCancelled)
+               .compose(this::compareDigests)
+
+               .andThen(ar -> {
+                   if (ar.failed())
+                   {
+                       LOGGER.error("{} Files verification task failed during validation", logPrefix, ar.cause());
+                       // Fail the task if it is in IN_PROGRESS state.
+                       state.compareAndSet(State.IN_PROGRESS, State.FAILED);
+                   }
+                   else
+                   {
+                       State currentState = state.compareAndExchange(State.IN_PROGRESS, State.COMPLETED);
+                       if (currentState == State.CANCELLED)
+                       {
+                           LOGGER.debug("{} Task was cancelled before completion", logPrefix);
+                       }
+                       else if (currentState != State.IN_PROGRESS)
+                       {
+                           LOGGER.warn("{} Unexpected state transition failure. State was {}", logPrefix, currentState);
+                       }
+                       else
+                       {
+                           LOGGER.info("{} Files verification task completed successfully", logPrefix);
+                       }
+                   }
+               });
+    }
+
+    private Future<List<InstanceFileInfo>> listFilesInLocal()
+    {
+        return executorPools.internal()
+                            .executeBlocking(() -> new CassandraInstanceFilesImpl(instanceMetadata,
+                                                                                  liveMigrationConfiguration)
+                                                   .files());
+    }
+
+    private Future<InstanceFilesListResponse> fetchSourceFileList()
+    {
+        return Future.fromCompletionStage(
+                     sidecarClient.liveMigrationListInstanceFilesAsync(new SidecarInstanceImpl(source, port)))
+                     .onFailure(cause -> LOGGER.error("{} Failed to obtain list of files from source {}:{}",
+                                                      logPrefix, source, port, cause));
+    }
+
+    private @NotNull Future<List<InstanceFileInfo>> compareFilesMeta(List<InstanceFileInfo> localFiles,

Review Comment:
   done



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationFilesVerificationTask.java:
##########
@@ -0,0 +1,643 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.Callable;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.vertx.core.Future;
+import io.vertx.core.MultiMap;
+import io.vertx.core.Promise;
+import io.vertx.core.Vertx;
+import org.apache.cassandra.sidecar.client.SidecarClient;
+import org.apache.cassandra.sidecar.client.SidecarInstanceImpl;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.DataObjectBuilder;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.request.data.Digest;
+import org.apache.cassandra.sidecar.common.request.data.MD5Digest;
+import org.apache.cassandra.sidecar.common.request.data.XXHash32Digest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.common.response.InstanceFileInfo;
+import org.apache.cassandra.sidecar.common.response.InstanceFilesListResponse;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.AsyncConcurrentTaskExecutor;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.DigestMismatchException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.FileVerificationFailureException;
+import org.apache.cassandra.sidecar.utils.DigestVerifierFactory;
+import org.jetbrains.annotations.NotNull;
+import org.jetbrains.annotations.VisibleForTesting;
+
+import static org.apache.cassandra.sidecar.common.response.InstanceFileInfo.FileType.DIRECTORY;
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationInstanceMetadataUtil.localPath;
+
+/**
+ * Verifies file integrity between source and destination instances during live migration
+ * by comparing file lists and validating file digests using configurable digest algorithms.
+ *
+ * <p>The verification process consists of three stages:
+ * <ol>
+ *   <li>Fetch file lists from both source and destination instances concurrently</li>
+ *   <li>Compare file metadata (size, type, modification time) - fails fast on mismatches</li>
+ *   <li>Verify cryptographic digests (MD5 or XXHash32) with configurable concurrency</li>
+ * </ol>
+ *
+ * <p>The task supports cancellation, and tracks detailed metrics including
+ * metadata matches/mismatches, digest verification results, and failure counts.
+ */
+public class LiveMigrationFilesVerificationTask implements LiveMigrationTask<LiveMigrationFilesVerificationResponse>
+{
+    public static final String FILES_VERIFICATION_TASK_TYPE = "files-verification-task";
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationFilesVerificationTask.class);
+    private final Vertx vertx;
+    private final String id;
+    private final String source;
+    private final int port;
+    // Instance metadata of current instance
+    private final InstanceMetadata instanceMetadata;
+    private final ExecutorPools executorPools;
+    private final LiveMigrationFilesVerificationRequest request;
+    private final SidecarClient sidecarClient;
+    private final LiveMigrationConfiguration liveMigrationConfiguration;
+    private final DigestVerifierFactory digestVerifierFactory;
+    private final AtomicReference<State> state;
+    private final Promise<Void> completionPromise = Promise.promise();
+    private final AtomicInteger filesNotFoundAtSource = new AtomicInteger(0);
+    private final AtomicInteger filesNotFoundAtDestination = new AtomicInteger(0);
+    private final AtomicInteger metadataMatched = new AtomicInteger(0);
+    private final AtomicInteger metadataMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestVerificationFailures = new AtomicInteger(0);
+    private final AtomicInteger filesMatched = new AtomicInteger(0);
+    private final String logPrefix;
+    private final AtomicBoolean cancelled = new AtomicBoolean(false);
+    private volatile AsyncConcurrentTaskExecutor<String> asyncConcurrentTaskExecutor;
+
+    @VisibleForTesting
+    protected LiveMigrationFilesVerificationTask(Builder builder)
+    {
+        this.vertx = builder.vertx;
+        this.id = builder.id;
+        this.source = builder.source;
+        this.port = builder.port;
+        this.instanceMetadata = builder.instanceMetadata;
+        this.executorPools = builder.executorPools;
+        this.request = builder.request;
+        this.sidecarClient = builder.sidecarClient;
+        this.liveMigrationConfiguration = builder.liveMigrationConfiguration;
+        this.digestVerifierFactory = builder.digestVerifierFactory;
+        this.logPrefix = "fileVerifyTaskId=" + id;
+        this.state = new AtomicReference<>(State.NOT_STARTED);
+    }
+
+    public static Builder builder()
+    {
+        return new Builder();
+    }
+
+    public Future<Void> validate()
+    {
+        return abortIfCancelled(new Object())
+               // Get files info from both source and destination
+               .compose(v -> Future.join(listFilesInLocal(), fetchSourceFileList()))
+
+               // Compare the files information before calculating digests
+               .compose(this::abortIfCancelled)
+               .compose(cf -> compareFilesMeta(cf.resultAt(0), cf.resultAt(1)))
+
+               // Create digest comparison tasks
+               .compose(this::abortIfCancelled)
+               .compose(this::getDigestComparisonTasks)
+
+               // Compare digests
+               .compose(this::abortIfCancelled)
+               .compose(this::compareDigests)
+
+               .andThen(ar -> {
+                   if (ar.failed())
+                   {
+                       LOGGER.error("{} Files verification task failed during validation", logPrefix, ar.cause());
+                       // Fail the task if it is in IN_PROGRESS state.
+                       state.compareAndSet(State.IN_PROGRESS, State.FAILED);
+                   }
+                   else
+                   {
+                       State currentState = state.compareAndExchange(State.IN_PROGRESS, State.COMPLETED);
+                       if (currentState == State.CANCELLED)
+                       {
+                           LOGGER.debug("{} Task was cancelled before completion", logPrefix);
+                       }
+                       else if (currentState != State.IN_PROGRESS)
+                       {
+                           LOGGER.warn("{} Unexpected state transition failure. State was {}", logPrefix, currentState);
+                       }
+                       else
+                       {
+                           LOGGER.info("{} Files verification task completed successfully", logPrefix);
+                       }
+                   }
+               });
+    }
+
+    private Future<List<InstanceFileInfo>> listFilesInLocal()
+    {
+        return executorPools.internal()
+                            .executeBlocking(() -> new CassandraInstanceFilesImpl(instanceMetadata,
+                                                                                  liveMigrationConfiguration)
+                                                   .files());
+    }
+
+    private Future<InstanceFilesListResponse> fetchSourceFileList()
+    {
+        return Future.fromCompletionStage(
+                     sidecarClient.liveMigrationListInstanceFilesAsync(new SidecarInstanceImpl(source, port)))
+                     .onFailure(cause -> LOGGER.error("{} Failed to obtain list of files from source {}:{}",
+                                                      logPrefix, source, port, cause));
+    }
+
+    private @NotNull Future<List<InstanceFileInfo>> compareFilesMeta(List<InstanceFileInfo> localFiles,
+                                                                     InstanceFilesListResponse sourceFiles)
+    {
+        Map<String, InstanceFileInfo> filesAtLocal =
+        localFiles.stream()
+                  .collect(Collectors.toMap(fileInfo -> fileInfo.fileUrl, fileInfo -> fileInfo));
+
+        Map<String, InstanceFileInfo> filesAtSource =
+        sourceFiles.getFiles().stream()
+                   .collect(Collectors.toMap(fileInfo -> fileInfo.fileUrl, fileInfo -> fileInfo));
+
+        for (Map.Entry<String, InstanceFileInfo> localFileEntry : filesAtLocal.entrySet())
+        {
+            String fileUrl = localFileEntry.getKey();
+            InstanceFileInfo localFile = localFileEntry.getValue();
+            if (filesAtSource.containsKey(fileUrl))
+            {
+                InstanceFileInfo sourceFile = filesAtSource.get(fileUrl);
+                // compare files metadata
+                if (localFile.equals(sourceFile) ||
+                    (localFile.fileType == DIRECTORY && sourceFile.fileType == DIRECTORY))
+                {
+                    // File info matches if either of them are directories or
+                    // their size, last modified time etc... matches
+                    LOGGER.debug("{} {} file metadata matched with source", logPrefix, fileUrl);
+                    metadataMatched.incrementAndGet();
+                }
+                else
+                {
+                    // Files metadata did not match
+                    metadataMismatches.incrementAndGet();
+                    LOGGER.error("{} {} did not match with source. Local file info={}, source file info={}",
+                                 logPrefix, fileUrl, localFile, sourceFile);
+                }
+
+                // done with processing current local file, remove it
+                filesAtSource.remove(fileUrl);
+            }
+            else
+            {
+                // File is not available at source, report it
+                LOGGER.error("{} File {} exists at destination but not found at source", logPrefix, fileUrl);
+                filesNotFoundAtSource.incrementAndGet();
+            }
+        }
+
+        for (Map.Entry<String, InstanceFileInfo> sourceFileEntry : filesAtSource.entrySet())
+        {
+            // Remaining files are not present at destination, report error for them
+            LOGGER.error("{} File {} exists at source but not found at destination", logPrefix, sourceFileEntry.getKey());
+            filesNotFoundAtDestination.incrementAndGet();
+        }
+
+        if (filesNotFoundAtDestination.get() > 0
+            || filesNotFoundAtSource.get() > 0
+            || metadataMismatches.get() > 0)
+        {
+            FileVerificationFailureException exception =
+            new FileVerificationFailureException("Files list did not match between source and destination");
+
+            return Future.failedFuture(exception);
+        }
+
+        return Future.succeededFuture(localFiles);
+    }
+
+    private @NotNull Future<List<Future<String>>> getDigestComparisonTasks(List<InstanceFileInfo> localFiles)
+    {
+        // now verify digests of each local file with source file
+        List<Callable<Future<String>>> tasks = new ArrayList<>(localFiles.size());
+        localFiles.stream()
+                  .filter(fileInfo -> fileInfo.fileType != DIRECTORY)
+                  .forEach(fileInfo -> tasks.add(() -> verifyDigest(fileInfo)));
+        asyncConcurrentTaskExecutor = new AsyncConcurrentTaskExecutor<>(vertx, tasks, request.maxConcurrency());
+        List<Future<String>> verifyTaskResults = asyncConcurrentTaskExecutor.start();
+        return Future.join(verifyTaskResults)
+                     .transform(ar -> Future.succeededFuture(verifyTaskResults));
+    }
+
+    private Future<Void> compareDigests(List<Future<String>> futures)
+    {
+        boolean verificationFailed = false;
+        for (Future<String> future : futures)
+        {
+            if (future.failed())
+            {
+                verificationFailed = true;
+                Throwable cause = future.cause();
+                if (cause instanceof DigestMismatchException)
+                {
+                    DigestMismatchException ex = (DigestMismatchException) cause;
+                    LOGGER.error("{} File digests did not match for {}.", logPrefix, ex.fileUrl(), cause);
+                    digestMismatches.incrementAndGet();
+                }
+                else
+                {
+                    LOGGER.error("{} Failed to verify digest for file. Check previous logs for file details",
+                                 logPrefix, cause);
+                    digestVerificationFailures.incrementAndGet();
+                }
+                continue;
+            }
+
+            // Digest verification succeeded
+            filesMatched.incrementAndGet();
+        }
+
+        if (verificationFailed)
+        {
+            return Future.failedFuture(
+            new FileVerificationFailureException("File digests did not match between source and destination"));
+        }
+        else
+        {
+            return Future.succeededFuture();
+        }
+    }
+
+    @VisibleForTesting
+    Future<String> verifyDigest(InstanceFileInfo fileInfo)
+    {
+        return getSourceFileDigest(fileInfo)
+               .compose(digest -> {
+                   String path = localPath(fileInfo.fileUrl, instanceMetadata).toAbsolutePath().toString();
+                   return digestVerifierFactory.verifier(MultiMap.caseInsensitiveMultiMap().addAll(digest.headers()))
+                                               .verify(path)
+                                               .compose(verified -> Future.succeededFuture(path))
+                                               .recover(cause -> Future.failedFuture(
+                                               new DigestMismatchException(path, fileInfo.fileUrl, cause)));
+               })
+               .onSuccess(filePath -> LOGGER.debug("{} Verified file {}", logPrefix, fileInfo.fileUrl))
+               .onFailure(cause -> LOGGER.error("{} Failed to verify file {}", logPrefix, fileInfo.fileUrl, cause));
+    }
+
+    private Future<Digest> getSourceFileDigest(InstanceFileInfo fileInfo)
+    {
+        return Future.fromCompletionStage(sidecarClient.liveMigrationFileDigestAsync(new SidecarInstanceImpl(source, port),
+                                                                                     fileInfo.fileUrl,
+                                                                                     request.digestAlgorithm()))
+                     .compose(this::toDigest);
+    }
+
+    private Future<Digest> toDigest(DigestResponse digestResponse)
+    {
+        String digestAlgorithm = digestResponse.digestAlgorithm;
+        if (digestAlgorithm.equalsIgnoreCase(MD5Digest.MD5_ALGORITHM))
+        {
+            return Future.succeededFuture(new MD5Digest(digestResponse.digest));
+        }
+        else if (digestAlgorithm.equalsIgnoreCase(XXHash32Digest.XXHASH_32_ALGORITHM))
+        {
+            return Future.succeededFuture(new XXHash32Digest(digestResponse.digest));
+        }

Review Comment:
   Felt both Digest and DigestResponse are not very relative to DigestAlgorithm. So, did not placed it DigestAlgorithmFactory. Moved it to DigestResponse.



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationTaskManager.java:
##########
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+
+import com.google.common.annotations.VisibleForTesting;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import org.apache.cassandra.sidecar.cluster.InstancesMetadata;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Centralized singleton manager for all live migration tasks across Cassandra instances.
+ * Enforces the constraint that only one {@link LiveMigrationTask} (of any type) can be active per instance at a time,
+ * preventing concurrent data copy and files verification operations that could lead to resource conflicts or data integrity issues.
+ * This singleton is shared across {@link DataCopyTaskManager} and {@link FilesVerificationTaskManager} to ensure
+ * proper coordination and mutual exclusion of tasks per instance.
+ */
+@Singleton
+public class LiveMigrationTaskManager
+{
+    @VisibleForTesting
+    final ConcurrentHashMap<Integer, LiveMigrationTask<?>> currentTasks = new ConcurrentHashMap<>();
+
+    private final InstancesMetadata instancesMetadata;
+
+    @Inject
+    public LiveMigrationTaskManager(InstancesMetadata instancesMetadata)
+    {
+        this.instancesMetadata = instancesMetadata;
+    }
+
+    /**
+     * Attempts to submit a new task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time.
+     *
+     * @param instanceId the instance ID
+     * @param newTask    the task to submit
+     * @return true if the task was accepted, false if another task is already in progress
+     */
+    public boolean submitTask(int instanceId, LiveMigrationTask<?> newTask)
+    {
+        return currentTasks.compute(instanceId, (integer, taskInMap) -> {
+            if (taskInMap == null)
+            {
+                return newTask;
+            }
+
+            if (!taskInMap.isCompleted())
+            {
+                // Reject new task if existing task is still in progress
+                return taskInMap;
+            }
+            else
+            {
+                // Accept new task if existing task has completed
+                return newTask;
+            }
+        }) == newTask;
+    }
+
+    /**
+     * Returns all live migration tasks for given currentHost.
+     * This includes both active and completed tasks that haven't been replaced.
+     *
+     * @param currentHost the host where sidecar is running
+     * @return list containing at most one task (empty if no task has ever been submitted for this host)
+     */
+    @SuppressWarnings("ConstantValue")
+    public List<LiveMigrationTask<?>> getAllTasks(@NotNull String currentHost)
+    {
+        InstanceMetadata localInstance = instancesMetadata.instanceFromHost(currentHost);
+        if (localInstance == null)
+        {
+            throw new IllegalStateException("No instance found for host: " + currentHost);
+        }
+        if (!currentTasks.containsKey(localInstance.id()))
+        {
+            return Collections.emptyList();
+        }
+        return Collections.singletonList(currentTasks.get(localInstance.id()));

Review Comment:
   Fetching task only once and returning the value.



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/FilesVerificationTaskManager.java:
##########
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.List;
+import java.util.Objects;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.datastax.driver.core.utils.UUIDs;
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import io.vertx.core.Future;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationInvalidRequestException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationFilesVerificationTask.FILES_VERIFICATION_TASK_TYPE;
+
+/**
+ * Manages the lifecycle of file digest verification tasks during live migration operations.
+ * This manager ensures that only one {@link LiveMigrationTask} can be active per instance at a time,
+ * preventing concurrent migration operations that could impact system resources.
+ * Tasks are created using the {@link LiveMigrationFilesVerificationTaskFactory} and
+ * executed asynchronously to validate file integrity between source and destination nodes.
+ */
+@Singleton
+public class FilesVerificationTaskManager
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(FilesVerificationTaskManager.class);
+
+    private final LiveMigrationTaskManager taskManager;
+    private final LiveMigrationFilesVerificationTaskFactory taskFactory;
+    private final SidecarConfiguration sidecarConfiguration;
+
+    @Inject
+    public FilesVerificationTaskManager(LiveMigrationTaskManager taskManager,
+                                        SidecarConfiguration sidecarConfiguration,
+                                        LiveMigrationFilesVerificationTaskFactory taskFactory)
+    {
+        this.taskManager = taskManager;
+        this.sidecarConfiguration = sidecarConfiguration;
+        this.taskFactory = taskFactory;
+    }
+
+    /**
+     * Creates and submits a new file digest verification task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time. If a task is already running,
+     * this method returns a failed future with {@link LiveMigrationTaskInProgressException}.
+     *
+     * @param request               the file verification request containing digest information
+     * @param source                the source identifier for the verification request
+     * @param localInstanceMetadata metadata of the local Cassandra instance
+     * @return a future that succeeds if the task is created and started, or fails with
+     * {@link LiveMigrationTaskInProgressException} if another task is already in progress
+     */
+    public Future<LiveMigrationTask<LiveMigrationFilesVerificationResponse>> createTask(LiveMigrationFilesVerificationRequest request,
+                                                                                        String source,
+                                                                                        InstanceMetadata localInstanceMetadata)
+    {
+        int maxPossibleConcurrency = Objects.requireNonNull(sidecarConfiguration.liveMigrationConfiguration())
+                                            .maxConcurrentFileRequests();
+        if (request.maxConcurrency() > maxPossibleConcurrency)
+        {
+            return Future.failedFuture(
+            new LiveMigrationInvalidRequestException("max concurrency can not be more than " + maxPossibleConcurrency));
+        }
+
+        return createVerifier(request, source, localInstanceMetadata)
+               .compose(newTask -> {
+                   boolean accepted = taskManager.submitTask(localInstanceMetadata.id(), newTask);
+
+                   if (accepted)
+                   {
+                       newTask.start();
+                       LOGGER.info("Accepted new files digest verification task for instance={} taskId={}",
+                                   localInstanceMetadata.id(), newTask.id());
+                       return Future.succeededFuture(newTask);
+                   }
+                   else
+                   {
+                       return Future.failedFuture(new LiveMigrationTaskInProgressException(
+                       "Another files digests verification is in progress for instance=" + localInstanceMetadata.id()));

Review Comment:
   done



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/FilesVerificationTaskManager.java:
##########
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.List;
+import java.util.Objects;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.datastax.driver.core.utils.UUIDs;
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import io.vertx.core.Future;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationInvalidRequestException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationFilesVerificationTask.FILES_VERIFICATION_TASK_TYPE;
+
+/**
+ * Manages the lifecycle of file digest verification tasks during live migration operations.
+ * This manager ensures that only one {@link LiveMigrationTask} can be active per instance at a time,
+ * preventing concurrent migration operations that could impact system resources.
+ * Tasks are created using the {@link LiveMigrationFilesVerificationTaskFactory} and
+ * executed asynchronously to validate file integrity between source and destination nodes.
+ */
+@Singleton
+public class FilesVerificationTaskManager

Review Comment:
   Started with inheriting LiveMigrationTaskManager, but had to switch to association. Since FilesVerificationTaskManager and DataCopyTaskManager are different types, Guice creates a separate instance for each, meaning each would get its own currentTasks map (instance to task map in LiveMigrationTaskManager). This breaks the invariant that only one task of any type can be active per instance at a time (line 44).  Don't want to use a static mutable map for inheritance. So, used association.



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/FilesVerificationTaskManager.java:
##########
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.List;
+import java.util.Objects;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.datastax.driver.core.utils.UUIDs;
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import io.vertx.core.Future;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationInvalidRequestException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationFilesVerificationTask.FILES_VERIFICATION_TASK_TYPE;
+
+/**
+ * Manages the lifecycle of file digest verification tasks during live migration operations.
+ * This manager ensures that only one {@link LiveMigrationTask} can be active per instance at a time,
+ * preventing concurrent migration operations that could impact system resources.
+ * Tasks are created using the {@link LiveMigrationFilesVerificationTaskFactory} and
+ * executed asynchronously to validate file integrity between source and destination nodes.
+ */
+@Singleton
+public class FilesVerificationTaskManager
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(FilesVerificationTaskManager.class);
+
+    private final LiveMigrationTaskManager taskManager;
+    private final LiveMigrationFilesVerificationTaskFactory taskFactory;
+    private final SidecarConfiguration sidecarConfiguration;
+
+    @Inject
+    public FilesVerificationTaskManager(LiveMigrationTaskManager taskManager,
+                                        SidecarConfiguration sidecarConfiguration,
+                                        LiveMigrationFilesVerificationTaskFactory taskFactory)
+    {
+        this.taskManager = taskManager;
+        this.sidecarConfiguration = sidecarConfiguration;
+        this.taskFactory = taskFactory;
+    }
+
+    /**
+     * Creates and submits a new file digest verification task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time. If a task is already running,
+     * this method returns a failed future with {@link LiveMigrationTaskInProgressException}.
+     *
+     * @param request               the file verification request containing digest information
+     * @param source                the source identifier for the verification request
+     * @param localInstanceMetadata metadata of the local Cassandra instance
+     * @return a future that succeeds if the task is created and started, or fails with
+     * {@link LiveMigrationTaskInProgressException} if another task is already in progress
+     */
+    public Future<LiveMigrationTask<LiveMigrationFilesVerificationResponse>> createTask(LiveMigrationFilesVerificationRequest request,
+                                                                                        String source,
+                                                                                        InstanceMetadata localInstanceMetadata)
+    {
+        int maxPossibleConcurrency = Objects.requireNonNull(sidecarConfiguration.liveMigrationConfiguration())
+                                            .maxConcurrentFileRequests();
+        if (request.maxConcurrency() > maxPossibleConcurrency)
+        {
+            return Future.failedFuture(
+            new LiveMigrationInvalidRequestException("max concurrency can not be more than " + maxPossibleConcurrency));
+        }
+
+        return createVerifier(request, source, localInstanceMetadata)
+               .compose(newTask -> {
+                   boolean accepted = taskManager.submitTask(localInstanceMetadata.id(), newTask);
+
+                   if (accepted)
+                   {
+                       newTask.start();
+                       LOGGER.info("Accepted new files digest verification task for instance={} taskId={}",
+                                   localInstanceMetadata.id(), newTask.id());
+                       return Future.succeededFuture(newTask);
+                   }
+                   else
+                   {
+                       return Future.failedFuture(new LiveMigrationTaskInProgressException(
+                       "Another files digests verification is in progress for instance=" + localInstanceMetadata.id()));
+                   }
+               });
+    }
+
+    private Future<LiveMigrationTask<LiveMigrationFilesVerificationResponse>> createVerifier(LiveMigrationFilesVerificationRequest request,
+                                                                                             String source,
+                                                                                             InstanceMetadata localInstanceMetadata)
+    {
+        String timeUuid = UUIDs.timeBased().toString();
+        return Future.succeededFuture(taskFactory.create(timeUuid,
+                                                         source,
+                                                         sidecarConfiguration.serviceConfiguration().port(),
+                                                         request,
+                                                         localInstanceMetadata));
+    }

Review Comment:
   make sense, updated it



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationFilesVerificationTask.java:
##########
@@ -0,0 +1,643 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.Callable;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.vertx.core.Future;
+import io.vertx.core.MultiMap;
+import io.vertx.core.Promise;
+import io.vertx.core.Vertx;
+import org.apache.cassandra.sidecar.client.SidecarClient;
+import org.apache.cassandra.sidecar.client.SidecarInstanceImpl;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.DataObjectBuilder;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.request.data.Digest;
+import org.apache.cassandra.sidecar.common.request.data.MD5Digest;
+import org.apache.cassandra.sidecar.common.request.data.XXHash32Digest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.common.response.InstanceFileInfo;
+import org.apache.cassandra.sidecar.common.response.InstanceFilesListResponse;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.AsyncConcurrentTaskExecutor;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.DigestMismatchException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.FileVerificationFailureException;
+import org.apache.cassandra.sidecar.utils.DigestVerifierFactory;
+import org.jetbrains.annotations.NotNull;
+import org.jetbrains.annotations.VisibleForTesting;
+
+import static org.apache.cassandra.sidecar.common.response.InstanceFileInfo.FileType.DIRECTORY;
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationInstanceMetadataUtil.localPath;
+
+/**
+ * Verifies file integrity between source and destination instances during live migration
+ * by comparing file lists and validating file digests using configurable digest algorithms.
+ *
+ * <p>The verification process consists of three stages:
+ * <ol>
+ *   <li>Fetch file lists from both source and destination instances concurrently</li>
+ *   <li>Compare file metadata (size, type, modification time) - fails fast on mismatches</li>
+ *   <li>Verify cryptographic digests (MD5 or XXHash32) with configurable concurrency</li>
+ * </ol>
+ *
+ * <p>The task supports cancellation, and tracks detailed metrics including
+ * metadata matches/mismatches, digest verification results, and failure counts.
+ */
+public class LiveMigrationFilesVerificationTask implements LiveMigrationTask<LiveMigrationFilesVerificationResponse>
+{
+    public static final String FILES_VERIFICATION_TASK_TYPE = "files-verification-task";
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationFilesVerificationTask.class);
+    private final Vertx vertx;
+    private final String id;
+    private final String source;
+    private final int port;
+    // Instance metadata of current instance
+    private final InstanceMetadata instanceMetadata;
+    private final ExecutorPools executorPools;
+    private final LiveMigrationFilesVerificationRequest request;
+    private final SidecarClient sidecarClient;
+    private final LiveMigrationConfiguration liveMigrationConfiguration;
+    private final DigestVerifierFactory digestVerifierFactory;
+    private final AtomicReference<State> state;
+    private final Promise<Void> completionPromise = Promise.promise();
+    private final AtomicInteger filesNotFoundAtSource = new AtomicInteger(0);
+    private final AtomicInteger filesNotFoundAtDestination = new AtomicInteger(0);
+    private final AtomicInteger metadataMatched = new AtomicInteger(0);
+    private final AtomicInteger metadataMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestVerificationFailures = new AtomicInteger(0);
+    private final AtomicInteger filesMatched = new AtomicInteger(0);
+    private final String logPrefix;
+    private final AtomicBoolean cancelled = new AtomicBoolean(false);
+    private volatile AsyncConcurrentTaskExecutor<String> asyncConcurrentTaskExecutor;
+
+    @VisibleForTesting
+    protected LiveMigrationFilesVerificationTask(Builder builder)
+    {
+        this.vertx = builder.vertx;
+        this.id = builder.id;
+        this.source = builder.source;
+        this.port = builder.port;
+        this.instanceMetadata = builder.instanceMetadata;
+        this.executorPools = builder.executorPools;
+        this.request = builder.request;
+        this.sidecarClient = builder.sidecarClient;
+        this.liveMigrationConfiguration = builder.liveMigrationConfiguration;
+        this.digestVerifierFactory = builder.digestVerifierFactory;
+        this.logPrefix = "fileVerifyTaskId=" + id;
+        this.state = new AtomicReference<>(State.NOT_STARTED);
+    }
+
+    public static Builder builder()
+    {
+        return new Builder();
+    }
+
+    public Future<Void> validate()
+    {
+        return abortIfCancelled(new Object())
+               // Get files info from both source and destination
+               .compose(v -> Future.join(listFilesInLocal(), fetchSourceFileList()))
+
+               // Compare the files information before calculating digests
+               .compose(this::abortIfCancelled)
+               .compose(cf -> compareFilesMeta(cf.resultAt(0), cf.resultAt(1)))
+
+               // Create digest comparison tasks
+               .compose(this::abortIfCancelled)
+               .compose(this::getDigestComparisonTasks)
+
+               // Compare digests
+               .compose(this::abortIfCancelled)
+               .compose(this::compareDigests)
+
+               .andThen(ar -> {
+                   if (ar.failed())
+                   {
+                       LOGGER.error("{} Files verification task failed during validation", logPrefix, ar.cause());
+                       // Fail the task if it is in IN_PROGRESS state.
+                       state.compareAndSet(State.IN_PROGRESS, State.FAILED);
+                   }
+                   else
+                   {
+                       State currentState = state.compareAndExchange(State.IN_PROGRESS, State.COMPLETED);
+                       if (currentState == State.CANCELLED)
+                       {
+                           LOGGER.debug("{} Task was cancelled before completion", logPrefix);
+                       }
+                       else if (currentState != State.IN_PROGRESS)
+                       {
+                           LOGGER.warn("{} Unexpected state transition failure. State was {}", logPrefix, currentState);
+                       }
+                       else
+                       {
+                           LOGGER.info("{} Files verification task completed successfully", logPrefix);
+                       }
+                   }
+               });
+    }
+
+    private Future<List<InstanceFileInfo>> listFilesInLocal()
+    {
+        return executorPools.internal()
+                            .executeBlocking(() -> new CassandraInstanceFilesImpl(instanceMetadata,
+                                                                                  liveMigrationConfiguration)
+                                                   .files());
+    }
+
+    private Future<InstanceFilesListResponse> fetchSourceFileList()
+    {
+        return Future.fromCompletionStage(
+                     sidecarClient.liveMigrationListInstanceFilesAsync(new SidecarInstanceImpl(source, port)))
+                     .onFailure(cause -> LOGGER.error("{} Failed to obtain list of files from source {}:{}",
+                                                      logPrefix, source, port, cause));
+    }
+
+    private @NotNull Future<List<InstanceFileInfo>> compareFilesMeta(List<InstanceFileInfo> localFiles,
+                                                                     InstanceFilesListResponse sourceFiles)
+    {
+        Map<String, InstanceFileInfo> filesAtLocal =
+        localFiles.stream()
+                  .collect(Collectors.toMap(fileInfo -> fileInfo.fileUrl, fileInfo -> fileInfo));
+
+        Map<String, InstanceFileInfo> filesAtSource =
+        sourceFiles.getFiles().stream()
+                   .collect(Collectors.toMap(fileInfo -> fileInfo.fileUrl, fileInfo -> fileInfo));
+
+        for (Map.Entry<String, InstanceFileInfo> localFileEntry : filesAtLocal.entrySet())
+        {
+            String fileUrl = localFileEntry.getKey();
+            InstanceFileInfo localFile = localFileEntry.getValue();
+            if (filesAtSource.containsKey(fileUrl))
+            {
+                InstanceFileInfo sourceFile = filesAtSource.get(fileUrl);
+                // compare files metadata
+                if (localFile.equals(sourceFile) ||
+                    (localFile.fileType == DIRECTORY && sourceFile.fileType == DIRECTORY))
+                {
+                    // File info matches if either of them are directories or
+                    // their size, last modified time etc... matches
+                    LOGGER.debug("{} {} file metadata matched with source", logPrefix, fileUrl);
+                    metadataMatched.incrementAndGet();
+                }
+                else
+                {
+                    // Files metadata did not match
+                    metadataMismatches.incrementAndGet();
+                    LOGGER.error("{} {} did not match with source. Local file info={}, source file info={}",
+                                 logPrefix, fileUrl, localFile, sourceFile);
+                }
+
+                // done with processing current local file, remove it
+                filesAtSource.remove(fileUrl);
+            }
+            else
+            {
+                // File is not available at source, report it
+                LOGGER.error("{} File {} exists at destination but not found at source", logPrefix, fileUrl);
+                filesNotFoundAtSource.incrementAndGet();
+            }
+        }
+
+        for (Map.Entry<String, InstanceFileInfo> sourceFileEntry : filesAtSource.entrySet())
+        {
+            // Remaining files are not present at destination, report error for them
+            LOGGER.error("{} File {} exists at source but not found at destination", logPrefix, sourceFileEntry.getKey());
+            filesNotFoundAtDestination.incrementAndGet();
+        }
+
+        if (filesNotFoundAtDestination.get() > 0
+            || filesNotFoundAtSource.get() > 0
+            || metadataMismatches.get() > 0)
+        {
+            FileVerificationFailureException exception =
+            new FileVerificationFailureException("Files list did not match between source and destination");
+
+            return Future.failedFuture(exception);
+        }
+
+        return Future.succeededFuture(localFiles);
+    }
+
+    private @NotNull Future<List<Future<String>>> getDigestComparisonTasks(List<InstanceFileInfo> localFiles)
+    {
+        // now verify digests of each local file with source file
+        List<Callable<Future<String>>> tasks = new ArrayList<>(localFiles.size());
+        localFiles.stream()
+                  .filter(fileInfo -> fileInfo.fileType != DIRECTORY)
+                  .forEach(fileInfo -> tasks.add(() -> verifyDigest(fileInfo)));
+        asyncConcurrentTaskExecutor = new AsyncConcurrentTaskExecutor<>(vertx, tasks, request.maxConcurrency());
+        List<Future<String>> verifyTaskResults = asyncConcurrentTaskExecutor.start();
+        return Future.join(verifyTaskResults)
+                     .transform(ar -> Future.succeededFuture(verifyTaskResults));
+    }
+
+    private Future<Void> compareDigests(List<Future<String>> futures)
+    {
+        boolean verificationFailed = false;
+        for (Future<String> future : futures)
+        {
+            if (future.failed())
+            {
+                verificationFailed = true;
+                Throwable cause = future.cause();
+                if (cause instanceof DigestMismatchException)
+                {
+                    DigestMismatchException ex = (DigestMismatchException) cause;
+                    LOGGER.error("{} File digests did not match for {}.", logPrefix, ex.fileUrl(), cause);
+                    digestMismatches.incrementAndGet();
+                }
+                else
+                {
+                    LOGGER.error("{} Failed to verify digest for file. Check previous logs for file details",
+                                 logPrefix, cause);
+                    digestVerificationFailures.incrementAndGet();
+                }
+                continue;
+            }
+
+            // Digest verification succeeded
+            filesMatched.incrementAndGet();
+        }
+
+        if (verificationFailed)
+        {
+            return Future.failedFuture(
+            new FileVerificationFailureException("File digests did not match between source and destination"));
+        }
+        else
+        {
+            return Future.succeededFuture();
+        }
+    }
+
+    @VisibleForTesting
+    Future<String> verifyDigest(InstanceFileInfo fileInfo)
+    {
+        return getSourceFileDigest(fileInfo)
+               .compose(digest -> {
+                   String path = localPath(fileInfo.fileUrl, instanceMetadata).toAbsolutePath().toString();
+                   return digestVerifierFactory.verifier(MultiMap.caseInsensitiveMultiMap().addAll(digest.headers()))
+                                               .verify(path)
+                                               .compose(verified -> Future.succeededFuture(path))
+                                               .recover(cause -> Future.failedFuture(
+                                               new DigestMismatchException(path, fileInfo.fileUrl, cause)));
+               })
+               .onSuccess(filePath -> LOGGER.debug("{} Verified file {}", logPrefix, fileInfo.fileUrl))
+               .onFailure(cause -> LOGGER.error("{} Failed to verify file {}", logPrefix, fileInfo.fileUrl, cause));
+    }
+
+    private Future<Digest> getSourceFileDigest(InstanceFileInfo fileInfo)
+    {
+        return Future.fromCompletionStage(sidecarClient.liveMigrationFileDigestAsync(new SidecarInstanceImpl(source, port),
+                                                                                     fileInfo.fileUrl,
+                                                                                     request.digestAlgorithm()))
+                     .compose(this::toDigest);
+    }

Review Comment:
   After giving some more thought, I feel the 429 status code (TOO_MANY_REQUESTS) is not appropriate and the 503 (SERVICE UNAVAILABLE) is more appropriate as maxConcurrentFileRequests (used by `LiveMigrationConcurrencyLimitHandler`) is a server-side concurrency cap shared across all clients, and an individual client hitting this limit hasn't done anything wrong - the server is simply busy with other requests. 503 describes this situation. 
   
   SidecarclientProvider is using `ExponentialBackoffRetryPolicy` as the default retry policy, which does retries. If we suspect that the default retry policy can be overridden, then I can explicitly initiate an instance of ExponentialBackoffRetryPolicy and use it.
   
   What do you think?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSSIDECAR-226 Adding endpoint for verifying files post data copy during live migration [cassandra-sidecar]

Posted by "yifan-c (via GitHub)" <gi...@apache.org>.
yifan-c commented on code in PR #309:
URL: https://github.com/apache/cassandra-sidecar/pull/309#discussion_r2808177605


##########
client-common/src/main/java/org/apache/cassandra/sidecar/common/request/LiveMigrationFileDigestRequest.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.common.request;
+
+import java.util.Objects;
+
+import io.netty.handler.codec.http.HttpMethod;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+
+/**
+ * Represents a request to retrieve file digest for validation during live migration.
+ * Supports configurable digest algorithms and optional seed values for verification.
+ */
+public class LiveMigrationFileDigestRequest extends JsonRequest<DigestResponse>
+{
+    public static final String DIGEST_ALGORITHM_PARAM = "digestAlgorithm";
+    public static final String SEED_PARAM = "seed";
+
+    /**
+     * Private constructor for internal use by the factory method
+     *
+     * @param fullRequestURI the complete URI with query parameters
+     */
+    private LiveMigrationFileDigestRequest(String fullRequestURI)
+    {
+        super(fullRequestURI);
+    }
+
+    /**
+     * Creates a live migration file digest request with validation
+     *
+     * @param requestURI the base URI of the request
+     * @param digestAlgorithm the digest algorithm to use (e.g., "MD5", "SHA-256")
+     * @param seed       optional seed value for digest verification, may be null
+     * @return a new LiveMigrationFileDigestRequest instance
+     * @throws NullPointerException     if requestURI or digestAlgorithm is null
+     * @throws IllegalArgumentException if digestAlgorithm is empty
+     */
+    public static LiveMigrationFileDigestRequest create(String requestURI, String digestAlgorithm, Integer seed)
+    {
+        Objects.requireNonNull(requestURI, "requestURI cannot be null");
+        Objects.requireNonNull(digestAlgorithm, "digestAlgorithm cannot be null");
+        if (digestAlgorithm.isEmpty())
+        {
+            throw new IllegalArgumentException("digestAlgorithm cannot be empty");
+        }
+
+        String fullURI = seed != null
+                         ? String.format("%s?%s=%s&%s=%d", requestURI, DIGEST_ALGORITHM_PARAM, digestAlgorithm, SEED_PARAM, seed)

Review Comment:
   One learning from the RestoreJob work is that the custom seed does not provide benefit for data integrity validation, but only adds code complexity. I would just drop the support of custom seed support to simplify the implementation, and use the fixed seed 0, which also makes the client-server communication simpler. 
   Not strong on removing the seed support, but feel ideal to do so. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSSIDECAR-226 Adding endpoint for verifying files post data copy during live migration [cassandra-sidecar]

Posted by "nvharikrishna (via GitHub)" <gi...@apache.org>.
nvharikrishna commented on code in PR #309:
URL: https://github.com/apache/cassandra-sidecar/pull/309#discussion_r2813455683


##########
client-common/src/main/java/org/apache/cassandra/sidecar/common/request/LiveMigrationFileDigestRequest.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.common.request;
+
+import java.util.Objects;
+
+import io.netty.handler.codec.http.HttpMethod;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+
+/**
+ * Represents a request to retrieve file digest for validation during live migration.
+ * Supports configurable digest algorithms and optional seed values for verification.
+ */
+public class LiveMigrationFileDigestRequest extends JsonRequest<DigestResponse>
+{
+    public static final String DIGEST_ALGORITHM_PARAM = "digestAlgorithm";
+    public static final String SEED_PARAM = "seed";
+
+    /**
+     * Private constructor for internal use by the factory method
+     *
+     * @param fullRequestURI the complete URI with query parameters
+     */
+    private LiveMigrationFileDigestRequest(String fullRequestURI)
+    {
+        super(fullRequestURI);
+    }
+
+    /**
+     * Creates a live migration file digest request with validation
+     *
+     * @param requestURI the base URI of the request
+     * @param digestAlgorithm the digest algorithm to use (e.g., "MD5", "SHA-256")
+     * @param seed       optional seed value for digest verification, may be null
+     * @return a new LiveMigrationFileDigestRequest instance
+     * @throws NullPointerException     if requestURI or digestAlgorithm is null
+     * @throws IllegalArgumentException if digestAlgorithm is empty
+     */
+    public static LiveMigrationFileDigestRequest create(String requestURI, String digestAlgorithm, Integer seed)
+    {
+        Objects.requireNonNull(requestURI, "requestURI cannot be null");
+        Objects.requireNonNull(digestAlgorithm, "digestAlgorithm cannot be null");
+        if (digestAlgorithm.isEmpty())
+        {
+            throw new IllegalArgumentException("digestAlgorithm cannot be empty");
+        }
+
+        String fullURI = seed != null
+                         ? String.format("%s?%s=%s&%s=%d", requestURI, DIGEST_ALGORITHM_PARAM, digestAlgorithm, SEED_PARAM, seed)

Review Comment:
   I agree with the code complexity and simplifying code suggestions. I can remove the support for seed for live migration. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSSIDECAR-226 Adding endpoint for verifying files post data copy during live migration [cassandra-sidecar]

Posted by "frankgh (via GitHub)" <gi...@apache.org>.
frankgh merged PR #309:
URL: https://github.com/apache/cassandra-sidecar/pull/309


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSSIDECAR-226 Adding endpoint for verifying files post data copy during live migration [cassandra-sidecar]

Posted by "yifan-c (via GitHub)" <gi...@apache.org>.
yifan-c commented on code in PR #309:
URL: https://github.com/apache/cassandra-sidecar/pull/309#discussion_r2868542781


##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCreateFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.json.DecodeException;
+import io.vertx.core.json.Json;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationTaskCreationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.NoSuchCassandraInstanceException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.DigestAlgorithmFactory;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.ACCEPTED;
+import static io.netty.handler.codec.http.HttpResponseStatus.FORBIDDEN;
+import static io.netty.handler.codec.http.HttpResponseStatus.SERVICE_UNAVAILABLE;
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.common.ApiEndpointsV1.LIVE_MIGRATION_FILES_VERIFICATION_TASKS_ROUTE;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * HTTP handler for creating file digest verification tasks during live migration.
+ * Manages concurrent verification tasks per instance and orchestrates the verification process.
+ */
+public class LiveMigrationCreateFilesVerificationTaskHandler extends AbstractHandler<LiveMigrationFilesVerificationRequest> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationCreateFilesVerificationTaskHandler.class);
+
+    private final FilesVerificationTaskManager filesVerificationTaskManager;
+    private final LiveMigrationMap liveMigrationMap;
+    private final LiveMigrationConfiguration liveMigrationConfiguration;
+
+    /**
+     * Constructs a handler with the provided {@code metadataFetcher}
+     *
+     * @param metadataFetcher the interface to retrieve instance metadata
+     * @param executorPools   the executor pools for blocking executions
+     * @param validator       a validator instance to validate Cassandra-specific input
+     */
+    @Inject
+    protected LiveMigrationCreateFilesVerificationTaskHandler(InstanceMetadataFetcher metadataFetcher,
+                                                              ExecutorPools executorPools,
+                                                              CassandraInputValidator validator,
+                                                              LiveMigrationMap liveMigrationMap,
+                                                              FilesVerificationTaskManager filesVerificationTaskManager,
+                                                              SidecarConfiguration sidecarConfiguration)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.liveMigrationMap = liveMigrationMap;
+        this.filesVerificationTaskManager = filesVerificationTaskManager;
+        this.liveMigrationConfiguration = sidecarConfiguration.liveMigrationConfiguration();
+    }
+
+    @Override
+    protected LiveMigrationFilesVerificationRequest extractParamsOrThrow(RoutingContext context)
+    {
+        try
+        {
+            LiveMigrationFilesVerificationRequest request =
+            Json.decodeValue(context.body().buffer(), LiveMigrationFilesVerificationRequest.class);
+
+            if (request.maxConcurrency() > liveMigrationConfiguration.maxConcurrentFileRequests())
+            {
+                throw new IllegalArgumentException("Invalid maxConcurrency " + request.maxConcurrency() +
+                                                   ". It cannot be greater than " +
+                                                   liveMigrationConfiguration.maxConcurrentFileRequests());
+            }
+
+            // Validate digest algorithm without creating an instance
+            DigestAlgorithmFactory.validateAlgorithmName(request.digestAlgorithm());
+
+            return request;
+        }
+        catch (DecodeException decodeException)
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST,
+                                    "Failed to parse request body, please ensure that the request is valid.",
+                                    decodeException);
+        }
+        catch (IllegalArgumentException e)
+        {
+            throw wrapHttpException(HttpResponseStatus.BAD_REQUEST, e.getMessage(), e);
+        }
+    }
+
+    @Override
+    protected void handleInternal(RoutingContext context,
+                                  HttpServerRequest httpRequest,
+                                  @NotNull String host,
+                                  SocketAddress remoteAddress,
+                                  LiveMigrationFilesVerificationRequest request)
+    {
+        LOGGER.debug("Received files verification request for host {} with maxConcurrency {}",
+                     host, request.maxConcurrency());
+        InstanceMetadata localInstanceMetadata;
+        try
+        {
+            localInstanceMetadata = metadataFetcher.instance(host);
+        }
+        catch (NoSuchCassandraInstanceException e)
+        {
+            LOGGER.error("Failed to fetch instance metadata for host={}", host);
+            context.fail(wrapHttpException(SERVICE_UNAVAILABLE, e));
+            return;
+        }
+
+        liveMigrationMap.getSource(host)
+                        .compose(source -> filesVerificationTaskManager.createTask(request, source, localInstanceMetadata))
+
+                        .onSuccess(task -> {
+                            LOGGER.info("Created files verification task {} for host {}", task.id(), host);
+                            context.response().setStatusCode(ACCEPTED.code());
+                            String statusUrl = LIVE_MIGRATION_FILES_VERIFICATION_TASKS_ROUTE + "/" + task.id();
+                            context.json(new LiveMigrationTaskCreationResponse(task.id(), statusUrl));
+                        })
+
+                        .onFailure(throwable -> {
+                            if (throwable instanceof LiveMigrationTaskInProgressException)
+                            {
+                                LOGGER.error("Cannot start a new files verification task for host {} " +
+                                             "while another live migration task is in progress.", host);
+                                context.fail(wrapHttpException(FORBIDDEN, throwable.getMessage(), throwable));

Review Comment:
   Should the status code be 409 Conflict, instead of Forbidden? 
   Forbidden typically means no permission to perform an action, not the cause here. 



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/FilesVerificationTaskManager.java:
##########
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.List;
+import java.util.Objects;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.datastax.driver.core.utils.UUIDs;
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import io.vertx.core.Future;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationInvalidRequestException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationFilesVerificationTask.FILES_VERIFICATION_TASK_TYPE;
+
+/**
+ * Manages the lifecycle of file digest verification tasks during live migration operations.
+ * This manager ensures that only one {@link LiveMigrationTask} can be active per instance at a time,
+ * preventing concurrent migration operations that could impact system resources.
+ * Tasks are created using the {@link LiveMigrationFilesVerificationTaskFactory} and
+ * executed asynchronously to validate file integrity between source and destination nodes.
+ */
+@Singleton
+public class FilesVerificationTaskManager
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(FilesVerificationTaskManager.class);
+
+    private final LiveMigrationTaskManager taskManager;
+    private final LiveMigrationFilesVerificationTaskFactory taskFactory;
+    private final SidecarConfiguration sidecarConfiguration;
+
+    @Inject
+    public FilesVerificationTaskManager(LiveMigrationTaskManager taskManager,
+                                        SidecarConfiguration sidecarConfiguration,
+                                        LiveMigrationFilesVerificationTaskFactory taskFactory)
+    {
+        this.taskManager = taskManager;
+        this.sidecarConfiguration = sidecarConfiguration;
+        this.taskFactory = taskFactory;
+    }
+
+    /**
+     * Creates and submits a new file digest verification task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time. If a task is already running,
+     * this method returns a failed future with {@link LiveMigrationTaskInProgressException}.
+     *
+     * @param request               the file verification request containing digest information
+     * @param source                the source identifier for the verification request
+     * @param localInstanceMetadata metadata of the local Cassandra instance
+     * @return a future that succeeds if the task is created and started, or fails with
+     * {@link LiveMigrationTaskInProgressException} if another task is already in progress
+     */
+    public Future<LiveMigrationTask<LiveMigrationFilesVerificationResponse>> createTask(LiveMigrationFilesVerificationRequest request,
+                                                                                        String source,
+                                                                                        InstanceMetadata localInstanceMetadata)
+    {
+        int maxPossibleConcurrency = Objects.requireNonNull(sidecarConfiguration.liveMigrationConfiguration())
+                                            .maxConcurrentFileRequests();
+        if (request.maxConcurrency() > maxPossibleConcurrency)
+        {
+            return Future.failedFuture(
+            new LiveMigrationInvalidRequestException("max concurrency can not be more than " + maxPossibleConcurrency));
+        }
+
+        return createVerifier(request, source, localInstanceMetadata)
+               .compose(newTask -> {
+                   boolean accepted = taskManager.submitTask(localInstanceMetadata.id(), newTask);
+
+                   if (accepted)
+                   {
+                       newTask.start();
+                       LOGGER.info("Accepted new files digest verification task for instance={} taskId={}",
+                                   localInstanceMetadata.id(), newTask.id());
+                       return Future.succeededFuture(newTask);
+                   }
+                   else
+                   {
+                       return Future.failedFuture(new LiveMigrationTaskInProgressException(
+                       "Another files digests verification is in progress for instance=" + localInstanceMetadata.id()));
+                   }
+               });
+    }
+
+    private Future<LiveMigrationTask<LiveMigrationFilesVerificationResponse>> createVerifier(LiveMigrationFilesVerificationRequest request,
+                                                                                             String source,
+                                                                                             InstanceMetadata localInstanceMetadata)
+    {
+        String timeUuid = UUIDs.timeBased().toString();
+        return Future.succeededFuture(taskFactory.create(timeUuid,
+                                                         source,
+                                                         sidecarConfiguration.serviceConfiguration().port(),
+                                                         request,
+                                                         localInstanceMetadata));
+    }

Review Comment:
   It is a private method and synchronous. The only reason is to chain `compose`. I think you can avoid wrapping `Future`, and just return `LiveMigrationTask`. 



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationTaskManager.java:
##########
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+
+import com.google.common.annotations.VisibleForTesting;
+
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import org.apache.cassandra.sidecar.cluster.InstancesMetadata;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * Centralized singleton manager for all live migration tasks across Cassandra instances.
+ * Enforces the constraint that only one {@link LiveMigrationTask} (of any type) can be active per instance at a time,
+ * preventing concurrent data copy and files verification operations that could lead to resource conflicts or data integrity issues.
+ * This singleton is shared across {@link DataCopyTaskManager} and {@link FilesVerificationTaskManager} to ensure
+ * proper coordination and mutual exclusion of tasks per instance.
+ */
+@Singleton
+public class LiveMigrationTaskManager
+{
+    @VisibleForTesting
+    final ConcurrentHashMap<Integer, LiveMigrationTask<?>> currentTasks = new ConcurrentHashMap<>();
+
+    private final InstancesMetadata instancesMetadata;
+
+    @Inject
+    public LiveMigrationTaskManager(InstancesMetadata instancesMetadata)
+    {
+        this.instancesMetadata = instancesMetadata;
+    }
+
+    /**
+     * Attempts to submit a new task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time.
+     *
+     * @param instanceId the instance ID
+     * @param newTask    the task to submit
+     * @return true if the task was accepted, false if another task is already in progress
+     */
+    public boolean submitTask(int instanceId, LiveMigrationTask<?> newTask)
+    {
+        return currentTasks.compute(instanceId, (integer, taskInMap) -> {
+            if (taskInMap == null)
+            {
+                return newTask;
+            }
+
+            if (!taskInMap.isCompleted())
+            {
+                // Reject new task if existing task is still in progress
+                return taskInMap;
+            }
+            else
+            {
+                // Accept new task if existing task has completed
+                return newTask;
+            }
+        }) == newTask;
+    }
+
+    /**
+     * Returns all live migration tasks for given currentHost.
+     * This includes both active and completed tasks that haven't been replaced.
+     *
+     * @param currentHost the host where sidecar is running
+     * @return list containing at most one task (empty if no task has ever been submitted for this host)
+     */
+    @SuppressWarnings("ConstantValue")
+    public List<LiveMigrationTask<?>> getAllTasks(@NotNull String currentHost)
+    {
+        InstanceMetadata localInstance = instancesMetadata.instanceFromHost(currentHost);
+        if (localInstance == null)
+        {
+            throw new IllegalStateException("No instance found for host: " + currentHost);
+        }
+        if (!currentTasks.containsKey(localInstance.id()))
+        {
+            return Collections.emptyList();
+        }
+        return Collections.singletonList(currentTasks.get(localInstance.id()));

Review Comment:
   `localInstance.id()` could potentially be removed at this step due to race condition. Instead, let's get the value at line#99 and return based on whether value is null or not. 



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/FilesVerificationTaskManager.java:
##########
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.List;
+import java.util.Objects;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.datastax.driver.core.utils.UUIDs;
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import io.vertx.core.Future;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationInvalidRequestException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationFilesVerificationTask.FILES_VERIFICATION_TASK_TYPE;
+
+/**
+ * Manages the lifecycle of file digest verification tasks during live migration operations.
+ * This manager ensures that only one {@link LiveMigrationTask} can be active per instance at a time,
+ * preventing concurrent migration operations that could impact system resources.
+ * Tasks are created using the {@link LiveMigrationFilesVerificationTaskFactory} and
+ * executed asynchronously to validate file integrity between source and destination nodes.
+ */
+@Singleton
+public class FilesVerificationTaskManager
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(FilesVerificationTaskManager.class);
+
+    private final LiveMigrationTaskManager taskManager;
+    private final LiveMigrationFilesVerificationTaskFactory taskFactory;
+    private final SidecarConfiguration sidecarConfiguration;
+
+    @Inject
+    public FilesVerificationTaskManager(LiveMigrationTaskManager taskManager,
+                                        SidecarConfiguration sidecarConfiguration,
+                                        LiveMigrationFilesVerificationTaskFactory taskFactory)
+    {
+        this.taskManager = taskManager;
+        this.sidecarConfiguration = sidecarConfiguration;
+        this.taskFactory = taskFactory;
+    }
+
+    /**
+     * Creates and submits a new file digest verification task for the specified instance.
+     * Only one task (of any type) can be active per instance at a time. If a task is already running,
+     * this method returns a failed future with {@link LiveMigrationTaskInProgressException}.
+     *
+     * @param request               the file verification request containing digest information
+     * @param source                the source identifier for the verification request
+     * @param localInstanceMetadata metadata of the local Cassandra instance
+     * @return a future that succeeds if the task is created and started, or fails with
+     * {@link LiveMigrationTaskInProgressException} if another task is already in progress
+     */
+    public Future<LiveMigrationTask<LiveMigrationFilesVerificationResponse>> createTask(LiveMigrationFilesVerificationRequest request,
+                                                                                        String source,
+                                                                                        InstanceMetadata localInstanceMetadata)
+    {
+        int maxPossibleConcurrency = Objects.requireNonNull(sidecarConfiguration.liveMigrationConfiguration())
+                                            .maxConcurrentFileRequests();
+        if (request.maxConcurrency() > maxPossibleConcurrency)
+        {
+            return Future.failedFuture(
+            new LiveMigrationInvalidRequestException("max concurrency can not be more than " + maxPossibleConcurrency));
+        }
+
+        return createVerifier(request, source, localInstanceMetadata)
+               .compose(newTask -> {
+                   boolean accepted = taskManager.submitTask(localInstanceMetadata.id(), newTask);
+
+                   if (accepted)
+                   {
+                       newTask.start();
+                       LOGGER.info("Accepted new files digest verification task for instance={} taskId={}",
+                                   localInstanceMetadata.id(), newTask.id());
+                       return Future.succeededFuture(newTask);
+                   }
+                   else
+                   {
+                       return Future.failedFuture(new LiveMigrationTaskInProgressException(
+                       "Another files digests verification is in progress for instance=" + localInstanceMetadata.id()));

Review Comment:
   ```suggestion
                          "Another files digest verification is in progress for instance=" + localInstanceMetadata.id()));
   ```



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationFilesVerificationTask.java:
##########
@@ -0,0 +1,643 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.Callable;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.vertx.core.Future;
+import io.vertx.core.MultiMap;
+import io.vertx.core.Promise;
+import io.vertx.core.Vertx;
+import org.apache.cassandra.sidecar.client.SidecarClient;
+import org.apache.cassandra.sidecar.client.SidecarInstanceImpl;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.DataObjectBuilder;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.request.data.Digest;
+import org.apache.cassandra.sidecar.common.request.data.MD5Digest;
+import org.apache.cassandra.sidecar.common.request.data.XXHash32Digest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.common.response.InstanceFileInfo;
+import org.apache.cassandra.sidecar.common.response.InstanceFilesListResponse;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.AsyncConcurrentTaskExecutor;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.DigestMismatchException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.FileVerificationFailureException;
+import org.apache.cassandra.sidecar.utils.DigestVerifierFactory;
+import org.jetbrains.annotations.NotNull;
+import org.jetbrains.annotations.VisibleForTesting;
+
+import static org.apache.cassandra.sidecar.common.response.InstanceFileInfo.FileType.DIRECTORY;
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationInstanceMetadataUtil.localPath;
+
+/**
+ * Verifies file integrity between source and destination instances during live migration
+ * by comparing file lists and validating file digests using configurable digest algorithms.
+ *
+ * <p>The verification process consists of three stages:
+ * <ol>
+ *   <li>Fetch file lists from both source and destination instances concurrently</li>
+ *   <li>Compare file metadata (size, type, modification time) - fails fast on mismatches</li>
+ *   <li>Verify cryptographic digests (MD5 or XXHash32) with configurable concurrency</li>
+ * </ol>
+ *
+ * <p>The task supports cancellation, and tracks detailed metrics including
+ * metadata matches/mismatches, digest verification results, and failure counts.
+ */
+public class LiveMigrationFilesVerificationTask implements LiveMigrationTask<LiveMigrationFilesVerificationResponse>
+{
+    public static final String FILES_VERIFICATION_TASK_TYPE = "files-verification-task";
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationFilesVerificationTask.class);
+    private final Vertx vertx;
+    private final String id;
+    private final String source;
+    private final int port;
+    // Instance metadata of current instance
+    private final InstanceMetadata instanceMetadata;
+    private final ExecutorPools executorPools;
+    private final LiveMigrationFilesVerificationRequest request;
+    private final SidecarClient sidecarClient;
+    private final LiveMigrationConfiguration liveMigrationConfiguration;
+    private final DigestVerifierFactory digestVerifierFactory;
+    private final AtomicReference<State> state;
+    private final Promise<Void> completionPromise = Promise.promise();
+    private final AtomicInteger filesNotFoundAtSource = new AtomicInteger(0);
+    private final AtomicInteger filesNotFoundAtDestination = new AtomicInteger(0);
+    private final AtomicInteger metadataMatched = new AtomicInteger(0);
+    private final AtomicInteger metadataMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestVerificationFailures = new AtomicInteger(0);
+    private final AtomicInteger filesMatched = new AtomicInteger(0);
+    private final String logPrefix;
+    private final AtomicBoolean cancelled = new AtomicBoolean(false);
+    private volatile AsyncConcurrentTaskExecutor<String> asyncConcurrentTaskExecutor;
+
+    @VisibleForTesting
+    protected LiveMigrationFilesVerificationTask(Builder builder)
+    {
+        this.vertx = builder.vertx;
+        this.id = builder.id;
+        this.source = builder.source;
+        this.port = builder.port;
+        this.instanceMetadata = builder.instanceMetadata;
+        this.executorPools = builder.executorPools;
+        this.request = builder.request;
+        this.sidecarClient = builder.sidecarClient;
+        this.liveMigrationConfiguration = builder.liveMigrationConfiguration;
+        this.digestVerifierFactory = builder.digestVerifierFactory;
+        this.logPrefix = "fileVerifyTaskId=" + id;
+        this.state = new AtomicReference<>(State.NOT_STARTED);
+    }
+
+    public static Builder builder()
+    {
+        return new Builder();
+    }
+
+    public Future<Void> validate()
+    {
+        return abortIfCancelled(new Object())
+               // Get files info from both source and destination
+               .compose(v -> Future.join(listFilesInLocal(), fetchSourceFileList()))
+
+               // Compare the files information before calculating digests
+               .compose(this::abortIfCancelled)
+               .compose(cf -> compareFilesMeta(cf.resultAt(0), cf.resultAt(1)))
+
+               // Create digest comparison tasks
+               .compose(this::abortIfCancelled)
+               .compose(this::getDigestComparisonTasks)
+
+               // Compare digests
+               .compose(this::abortIfCancelled)
+               .compose(this::compareDigests)
+
+               .andThen(ar -> {
+                   if (ar.failed())
+                   {
+                       LOGGER.error("{} Files verification task failed during validation", logPrefix, ar.cause());
+                       // Fail the task if it is in IN_PROGRESS state.
+                       state.compareAndSet(State.IN_PROGRESS, State.FAILED);
+                   }
+                   else
+                   {
+                       State currentState = state.compareAndExchange(State.IN_PROGRESS, State.COMPLETED);
+                       if (currentState == State.CANCELLED)
+                       {
+                           LOGGER.debug("{} Task was cancelled before completion", logPrefix);
+                       }
+                       else if (currentState != State.IN_PROGRESS)
+                       {
+                           LOGGER.warn("{} Unexpected state transition failure. State was {}", logPrefix, currentState);
+                       }
+                       else
+                       {
+                           LOGGER.info("{} Files verification task completed successfully", logPrefix);
+                       }
+                   }
+               });
+    }
+
+    private Future<List<InstanceFileInfo>> listFilesInLocal()
+    {
+        return executorPools.internal()
+                            .executeBlocking(() -> new CassandraInstanceFilesImpl(instanceMetadata,
+                                                                                  liveMigrationConfiguration)
+                                                   .files());
+    }
+
+    private Future<InstanceFilesListResponse> fetchSourceFileList()
+    {
+        return Future.fromCompletionStage(
+                     sidecarClient.liveMigrationListInstanceFilesAsync(new SidecarInstanceImpl(source, port)))
+                     .onFailure(cause -> LOGGER.error("{} Failed to obtain list of files from source {}:{}",
+                                                      logPrefix, source, port, cause));
+    }
+
+    private @NotNull Future<List<InstanceFileInfo>> compareFilesMeta(List<InstanceFileInfo> localFiles,

Review Comment:
   nit: prefer `compareFilesMetadata`



##########
server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationCreateFilesVerificationTaskHandler.java:
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.handlers.livemigration;
+
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.inject.Inject;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.http.HttpServerRequest;
+import io.vertx.core.json.DecodeException;
+import io.vertx.core.json.Json;
+import io.vertx.core.net.SocketAddress;
+import io.vertx.ext.auth.authorization.Authorization;
+import io.vertx.ext.web.RoutingContext;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationTaskCreationResponse;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.NoSuchCassandraInstanceException;
+import org.apache.cassandra.sidecar.handlers.AbstractHandler;
+import org.apache.cassandra.sidecar.handlers.AccessProtected;
+import org.apache.cassandra.sidecar.livemigration.FilesVerificationTaskManager;
+import org.apache.cassandra.sidecar.utils.CassandraInputValidator;
+import org.apache.cassandra.sidecar.utils.DigestAlgorithmFactory;
+import org.apache.cassandra.sidecar.utils.InstanceMetadataFetcher;
+import org.jetbrains.annotations.NotNull;
+
+import static io.netty.handler.codec.http.HttpResponseStatus.ACCEPTED;
+import static io.netty.handler.codec.http.HttpResponseStatus.FORBIDDEN;
+import static io.netty.handler.codec.http.HttpResponseStatus.SERVICE_UNAVAILABLE;
+import static org.apache.cassandra.sidecar.acl.authorization.BasicPermissions.DATA_COPY;
+import static org.apache.cassandra.sidecar.common.ApiEndpointsV1.LIVE_MIGRATION_FILES_VERIFICATION_TASKS_ROUTE;
+import static org.apache.cassandra.sidecar.utils.HttpExceptions.wrapHttpException;
+
+/**
+ * HTTP handler for creating file digest verification tasks during live migration.
+ * Manages concurrent verification tasks per instance and orchestrates the verification process.
+ */
+public class LiveMigrationCreateFilesVerificationTaskHandler extends AbstractHandler<LiveMigrationFilesVerificationRequest> implements AccessProtected
+{
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationCreateFilesVerificationTaskHandler.class);
+
+    private final FilesVerificationTaskManager filesVerificationTaskManager;
+    private final LiveMigrationMap liveMigrationMap;
+    private final LiveMigrationConfiguration liveMigrationConfiguration;
+
+    /**
+     * Constructs a handler with the provided {@code metadataFetcher}
+     *
+     * @param metadataFetcher the interface to retrieve instance metadata
+     * @param executorPools   the executor pools for blocking executions
+     * @param validator       a validator instance to validate Cassandra-specific input
+     */
+    @Inject
+    protected LiveMigrationCreateFilesVerificationTaskHandler(InstanceMetadataFetcher metadataFetcher,
+                                                              ExecutorPools executorPools,
+                                                              CassandraInputValidator validator,
+                                                              LiveMigrationMap liveMigrationMap,
+                                                              FilesVerificationTaskManager filesVerificationTaskManager,
+                                                              SidecarConfiguration sidecarConfiguration)
+    {
+        super(metadataFetcher, executorPools, validator);
+        this.liveMigrationMap = liveMigrationMap;
+        this.filesVerificationTaskManager = filesVerificationTaskManager;
+        this.liveMigrationConfiguration = sidecarConfiguration.liveMigrationConfiguration();
+    }
+
+    @Override
+    protected LiveMigrationFilesVerificationRequest extractParamsOrThrow(RoutingContext context)
+    {
+        try
+        {
+            LiveMigrationFilesVerificationRequest request =
+            Json.decodeValue(context.body().buffer(), LiveMigrationFilesVerificationRequest.class);
+
+            if (request.maxConcurrency() > liveMigrationConfiguration.maxConcurrentFileRequests())
+            {
+                throw new IllegalArgumentException("Invalid maxConcurrency " + request.maxConcurrency() +
+                                                   ". It cannot be greater than " +
+                                                   liveMigrationConfiguration.maxConcurrentFileRequests());
+            }

Review Comment:
   In `FilesVerificationTaskManager`, it handles `maxConcurrency` differently. Can you address the inconsistency or the duplication? There seems to be sufficient to have one validation only. 
   
   ```
           if (request.maxConcurrency() > maxPossibleConcurrency)
           {
               return Future.failedFuture(
               new LiveMigrationInvalidRequestException("max concurrency can not be more than " + maxPossibleConcurrency));
           }
   ```



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationFilesVerificationTask.java:
##########
@@ -0,0 +1,643 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.Callable;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.vertx.core.Future;
+import io.vertx.core.MultiMap;
+import io.vertx.core.Promise;
+import io.vertx.core.Vertx;
+import org.apache.cassandra.sidecar.client.SidecarClient;
+import org.apache.cassandra.sidecar.client.SidecarInstanceImpl;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.DataObjectBuilder;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.request.data.Digest;
+import org.apache.cassandra.sidecar.common.request.data.MD5Digest;
+import org.apache.cassandra.sidecar.common.request.data.XXHash32Digest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.common.response.InstanceFileInfo;
+import org.apache.cassandra.sidecar.common.response.InstanceFilesListResponse;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.AsyncConcurrentTaskExecutor;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.DigestMismatchException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.FileVerificationFailureException;
+import org.apache.cassandra.sidecar.utils.DigestVerifierFactory;
+import org.jetbrains.annotations.NotNull;
+import org.jetbrains.annotations.VisibleForTesting;
+
+import static org.apache.cassandra.sidecar.common.response.InstanceFileInfo.FileType.DIRECTORY;
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationInstanceMetadataUtil.localPath;
+
+/**
+ * Verifies file integrity between source and destination instances during live migration
+ * by comparing file lists and validating file digests using configurable digest algorithms.
+ *
+ * <p>The verification process consists of three stages:
+ * <ol>
+ *   <li>Fetch file lists from both source and destination instances concurrently</li>
+ *   <li>Compare file metadata (size, type, modification time) - fails fast on mismatches</li>

Review Comment:
   I do not think `compareFilesMeta()` (again, prefer `compareFilesMetadata()`) fail fast. It loops through all files and collects all the errors. Please update the java doc to reflect the actual implementation. 



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationFilesVerificationTask.java:
##########
@@ -0,0 +1,643 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.Callable;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.vertx.core.Future;
+import io.vertx.core.MultiMap;
+import io.vertx.core.Promise;
+import io.vertx.core.Vertx;
+import org.apache.cassandra.sidecar.client.SidecarClient;
+import org.apache.cassandra.sidecar.client.SidecarInstanceImpl;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.DataObjectBuilder;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.request.data.Digest;
+import org.apache.cassandra.sidecar.common.request.data.MD5Digest;
+import org.apache.cassandra.sidecar.common.request.data.XXHash32Digest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.common.response.InstanceFileInfo;
+import org.apache.cassandra.sidecar.common.response.InstanceFilesListResponse;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.AsyncConcurrentTaskExecutor;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.DigestMismatchException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.FileVerificationFailureException;
+import org.apache.cassandra.sidecar.utils.DigestVerifierFactory;
+import org.jetbrains.annotations.NotNull;
+import org.jetbrains.annotations.VisibleForTesting;
+
+import static org.apache.cassandra.sidecar.common.response.InstanceFileInfo.FileType.DIRECTORY;
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationInstanceMetadataUtil.localPath;
+
+/**
+ * Verifies file integrity between source and destination instances during live migration
+ * by comparing file lists and validating file digests using configurable digest algorithms.
+ *
+ * <p>The verification process consists of three stages:
+ * <ol>
+ *   <li>Fetch file lists from both source and destination instances concurrently</li>
+ *   <li>Compare file metadata (size, type, modification time) - fails fast on mismatches</li>
+ *   <li>Verify cryptographic digests (MD5 or XXHash32) with configurable concurrency</li>
+ * </ol>
+ *
+ * <p>The task supports cancellation, and tracks detailed metrics including
+ * metadata matches/mismatches, digest verification results, and failure counts.
+ */
+public class LiveMigrationFilesVerificationTask implements LiveMigrationTask<LiveMigrationFilesVerificationResponse>
+{
+    public static final String FILES_VERIFICATION_TASK_TYPE = "files-verification-task";
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationFilesVerificationTask.class);
+    private final Vertx vertx;
+    private final String id;
+    private final String source;
+    private final int port;
+    // Instance metadata of current instance
+    private final InstanceMetadata instanceMetadata;
+    private final ExecutorPools executorPools;
+    private final LiveMigrationFilesVerificationRequest request;
+    private final SidecarClient sidecarClient;
+    private final LiveMigrationConfiguration liveMigrationConfiguration;
+    private final DigestVerifierFactory digestVerifierFactory;
+    private final AtomicReference<State> state;
+    private final Promise<Void> completionPromise = Promise.promise();
+    private final AtomicInteger filesNotFoundAtSource = new AtomicInteger(0);
+    private final AtomicInteger filesNotFoundAtDestination = new AtomicInteger(0);
+    private final AtomicInteger metadataMatched = new AtomicInteger(0);
+    private final AtomicInteger metadataMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestVerificationFailures = new AtomicInteger(0);
+    private final AtomicInteger filesMatched = new AtomicInteger(0);
+    private final String logPrefix;
+    private final AtomicBoolean cancelled = new AtomicBoolean(false);
+    private volatile AsyncConcurrentTaskExecutor<String> asyncConcurrentTaskExecutor;
+
+    @VisibleForTesting
+    protected LiveMigrationFilesVerificationTask(Builder builder)
+    {
+        this.vertx = builder.vertx;
+        this.id = builder.id;
+        this.source = builder.source;
+        this.port = builder.port;
+        this.instanceMetadata = builder.instanceMetadata;
+        this.executorPools = builder.executorPools;
+        this.request = builder.request;
+        this.sidecarClient = builder.sidecarClient;
+        this.liveMigrationConfiguration = builder.liveMigrationConfiguration;
+        this.digestVerifierFactory = builder.digestVerifierFactory;
+        this.logPrefix = "fileVerifyTaskId=" + id;
+        this.state = new AtomicReference<>(State.NOT_STARTED);
+    }
+
+    public static Builder builder()
+    {
+        return new Builder();
+    }
+
+    public Future<Void> validate()
+    {
+        return abortIfCancelled(new Object())
+               // Get files info from both source and destination
+               .compose(v -> Future.join(listFilesInLocal(), fetchSourceFileList()))
+
+               // Compare the files information before calculating digests
+               .compose(this::abortIfCancelled)
+               .compose(cf -> compareFilesMeta(cf.resultAt(0), cf.resultAt(1)))
+
+               // Create digest comparison tasks
+               .compose(this::abortIfCancelled)
+               .compose(this::getDigestComparisonTasks)
+
+               // Compare digests
+               .compose(this::abortIfCancelled)
+               .compose(this::compareDigests)
+
+               .andThen(ar -> {
+                   if (ar.failed())
+                   {
+                       LOGGER.error("{} Files verification task failed during validation", logPrefix, ar.cause());
+                       // Fail the task if it is in IN_PROGRESS state.
+                       state.compareAndSet(State.IN_PROGRESS, State.FAILED);
+                   }
+                   else
+                   {
+                       State currentState = state.compareAndExchange(State.IN_PROGRESS, State.COMPLETED);
+                       if (currentState == State.CANCELLED)
+                       {
+                           LOGGER.debug("{} Task was cancelled before completion", logPrefix);
+                       }
+                       else if (currentState != State.IN_PROGRESS)
+                       {
+                           LOGGER.warn("{} Unexpected state transition failure. State was {}", logPrefix, currentState);
+                       }
+                       else
+                       {
+                           LOGGER.info("{} Files verification task completed successfully", logPrefix);
+                       }
+                   }
+               });
+    }
+
+    private Future<List<InstanceFileInfo>> listFilesInLocal()
+    {
+        return executorPools.internal()
+                            .executeBlocking(() -> new CassandraInstanceFilesImpl(instanceMetadata,
+                                                                                  liveMigrationConfiguration)
+                                                   .files());
+    }
+
+    private Future<InstanceFilesListResponse> fetchSourceFileList()
+    {
+        return Future.fromCompletionStage(
+                     sidecarClient.liveMigrationListInstanceFilesAsync(new SidecarInstanceImpl(source, port)))
+                     .onFailure(cause -> LOGGER.error("{} Failed to obtain list of files from source {}:{}",
+                                                      logPrefix, source, port, cause));
+    }
+
+    private @NotNull Future<List<InstanceFileInfo>> compareFilesMeta(List<InstanceFileInfo> localFiles,
+                                                                     InstanceFilesListResponse sourceFiles)
+    {
+        Map<String, InstanceFileInfo> filesAtLocal =
+        localFiles.stream()
+                  .collect(Collectors.toMap(fileInfo -> fileInfo.fileUrl, fileInfo -> fileInfo));
+
+        Map<String, InstanceFileInfo> filesAtSource =
+        sourceFiles.getFiles().stream()
+                   .collect(Collectors.toMap(fileInfo -> fileInfo.fileUrl, fileInfo -> fileInfo));
+
+        for (Map.Entry<String, InstanceFileInfo> localFileEntry : filesAtLocal.entrySet())
+        {
+            String fileUrl = localFileEntry.getKey();
+            InstanceFileInfo localFile = localFileEntry.getValue();
+            if (filesAtSource.containsKey(fileUrl))
+            {
+                InstanceFileInfo sourceFile = filesAtSource.get(fileUrl);
+                // compare files metadata
+                if (localFile.equals(sourceFile) ||
+                    (localFile.fileType == DIRECTORY && sourceFile.fileType == DIRECTORY))
+                {
+                    // File info matches if either of them are directories or
+                    // their size, last modified time etc... matches
+                    LOGGER.debug("{} {} file metadata matched with source", logPrefix, fileUrl);
+                    metadataMatched.incrementAndGet();
+                }
+                else
+                {
+                    // Files metadata did not match
+                    metadataMismatches.incrementAndGet();
+                    LOGGER.error("{} {} did not match with source. Local file info={}, source file info={}",
+                                 logPrefix, fileUrl, localFile, sourceFile);
+                }
+
+                // done with processing current local file, remove it
+                filesAtSource.remove(fileUrl);
+            }
+            else
+            {
+                // File is not available at source, report it
+                LOGGER.error("{} File {} exists at destination but not found at source", logPrefix, fileUrl);
+                filesNotFoundAtSource.incrementAndGet();
+            }
+        }
+
+        for (Map.Entry<String, InstanceFileInfo> sourceFileEntry : filesAtSource.entrySet())
+        {
+            // Remaining files are not present at destination, report error for them
+            LOGGER.error("{} File {} exists at source but not found at destination", logPrefix, sourceFileEntry.getKey());
+            filesNotFoundAtDestination.incrementAndGet();
+        }
+
+        if (filesNotFoundAtDestination.get() > 0
+            || filesNotFoundAtSource.get() > 0
+            || metadataMismatches.get() > 0)
+        {
+            FileVerificationFailureException exception =
+            new FileVerificationFailureException("Files list did not match between source and destination");
+
+            return Future.failedFuture(exception);
+        }
+
+        return Future.succeededFuture(localFiles);
+    }
+
+    private @NotNull Future<List<Future<String>>> getDigestComparisonTasks(List<InstanceFileInfo> localFiles)
+    {
+        // now verify digests of each local file with source file
+        List<Callable<Future<String>>> tasks = new ArrayList<>(localFiles.size());
+        localFiles.stream()
+                  .filter(fileInfo -> fileInfo.fileType != DIRECTORY)
+                  .forEach(fileInfo -> tasks.add(() -> verifyDigest(fileInfo)));
+        asyncConcurrentTaskExecutor = new AsyncConcurrentTaskExecutor<>(vertx, tasks, request.maxConcurrency());
+        List<Future<String>> verifyTaskResults = asyncConcurrentTaskExecutor.start();
+        return Future.join(verifyTaskResults)
+                     .transform(ar -> Future.succeededFuture(verifyTaskResults));
+    }
+
+    private Future<Void> compareDigests(List<Future<String>> futures)
+    {
+        boolean verificationFailed = false;
+        for (Future<String> future : futures)
+        {
+            if (future.failed())
+            {
+                verificationFailed = true;
+                Throwable cause = future.cause();
+                if (cause instanceof DigestMismatchException)
+                {
+                    DigestMismatchException ex = (DigestMismatchException) cause;
+                    LOGGER.error("{} File digests did not match for {}.", logPrefix, ex.fileUrl(), cause);
+                    digestMismatches.incrementAndGet();
+                }
+                else
+                {
+                    LOGGER.error("{} Failed to verify digest for file. Check previous logs for file details",
+                                 logPrefix, cause);
+                    digestVerificationFailures.incrementAndGet();
+                }
+                continue;
+            }
+
+            // Digest verification succeeded
+            filesMatched.incrementAndGet();
+        }
+
+        if (verificationFailed)
+        {
+            return Future.failedFuture(
+            new FileVerificationFailureException("File digests did not match between source and destination"));
+        }
+        else
+        {
+            return Future.succeededFuture();
+        }
+    }
+
+    @VisibleForTesting
+    Future<String> verifyDigest(InstanceFileInfo fileInfo)
+    {
+        return getSourceFileDigest(fileInfo)
+               .compose(digest -> {
+                   String path = localPath(fileInfo.fileUrl, instanceMetadata).toAbsolutePath().toString();
+                   return digestVerifierFactory.verifier(MultiMap.caseInsensitiveMultiMap().addAll(digest.headers()))
+                                               .verify(path)
+                                               .compose(verified -> Future.succeededFuture(path))
+                                               .recover(cause -> Future.failedFuture(
+                                               new DigestMismatchException(path, fileInfo.fileUrl, cause)));
+               })
+               .onSuccess(filePath -> LOGGER.debug("{} Verified file {}", logPrefix, fileInfo.fileUrl))
+               .onFailure(cause -> LOGGER.error("{} Failed to verify file {}", logPrefix, fileInfo.fileUrl, cause));
+    }
+
+    private Future<Digest> getSourceFileDigest(InstanceFileInfo fileInfo)
+    {
+        return Future.fromCompletionStage(sidecarClient.liveMigrationFileDigestAsync(new SidecarInstanceImpl(source, port),
+                                                                                     fileInfo.fileUrl,
+                                                                                     request.digestAlgorithm()))
+                     .compose(this::toDigest);
+    }
+
+    private Future<Digest> toDigest(DigestResponse digestResponse)
+    {
+        String digestAlgorithm = digestResponse.digestAlgorithm;
+        if (digestAlgorithm.equalsIgnoreCase(MD5Digest.MD5_ALGORITHM))
+        {
+            return Future.succeededFuture(new MD5Digest(digestResponse.digest));
+        }
+        else if (digestAlgorithm.equalsIgnoreCase(XXHash32Digest.XXHASH_32_ALGORITHM))
+        {
+            return Future.succeededFuture(new XXHash32Digest(digestResponse.digest));
+        }

Review Comment:
   Should it be in `DigestAlgorithmFactory`? 



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/LiveMigrationFilesVerificationTask.java:
##########
@@ -0,0 +1,643 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.Callable;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.vertx.core.Future;
+import io.vertx.core.MultiMap;
+import io.vertx.core.Promise;
+import io.vertx.core.Vertx;
+import org.apache.cassandra.sidecar.client.SidecarClient;
+import org.apache.cassandra.sidecar.client.SidecarInstanceImpl;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.DataObjectBuilder;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.request.data.Digest;
+import org.apache.cassandra.sidecar.common.request.data.MD5Digest;
+import org.apache.cassandra.sidecar.common.request.data.XXHash32Digest;
+import org.apache.cassandra.sidecar.common.response.DigestResponse;
+import org.apache.cassandra.sidecar.common.response.InstanceFileInfo;
+import org.apache.cassandra.sidecar.common.response.InstanceFilesListResponse;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.concurrent.AsyncConcurrentTaskExecutor;
+import org.apache.cassandra.sidecar.concurrent.ExecutorPools;
+import org.apache.cassandra.sidecar.config.LiveMigrationConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.DigestMismatchException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.FileVerificationFailureException;
+import org.apache.cassandra.sidecar.utils.DigestVerifierFactory;
+import org.jetbrains.annotations.NotNull;
+import org.jetbrains.annotations.VisibleForTesting;
+
+import static org.apache.cassandra.sidecar.common.response.InstanceFileInfo.FileType.DIRECTORY;
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationInstanceMetadataUtil.localPath;
+
+/**
+ * Verifies file integrity between source and destination instances during live migration
+ * by comparing file lists and validating file digests using configurable digest algorithms.
+ *
+ * <p>The verification process consists of three stages:
+ * <ol>
+ *   <li>Fetch file lists from both source and destination instances concurrently</li>
+ *   <li>Compare file metadata (size, type, modification time) - fails fast on mismatches</li>
+ *   <li>Verify cryptographic digests (MD5 or XXHash32) with configurable concurrency</li>
+ * </ol>
+ *
+ * <p>The task supports cancellation, and tracks detailed metrics including
+ * metadata matches/mismatches, digest verification results, and failure counts.
+ */
+public class LiveMigrationFilesVerificationTask implements LiveMigrationTask<LiveMigrationFilesVerificationResponse>
+{
+    public static final String FILES_VERIFICATION_TASK_TYPE = "files-verification-task";
+    private static final Logger LOGGER = LoggerFactory.getLogger(LiveMigrationFilesVerificationTask.class);
+    private final Vertx vertx;
+    private final String id;
+    private final String source;
+    private final int port;
+    // Instance metadata of current instance
+    private final InstanceMetadata instanceMetadata;
+    private final ExecutorPools executorPools;
+    private final LiveMigrationFilesVerificationRequest request;
+    private final SidecarClient sidecarClient;
+    private final LiveMigrationConfiguration liveMigrationConfiguration;
+    private final DigestVerifierFactory digestVerifierFactory;
+    private final AtomicReference<State> state;
+    private final Promise<Void> completionPromise = Promise.promise();
+    private final AtomicInteger filesNotFoundAtSource = new AtomicInteger(0);
+    private final AtomicInteger filesNotFoundAtDestination = new AtomicInteger(0);
+    private final AtomicInteger metadataMatched = new AtomicInteger(0);
+    private final AtomicInteger metadataMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestMismatches = new AtomicInteger(0);
+    private final AtomicInteger digestVerificationFailures = new AtomicInteger(0);
+    private final AtomicInteger filesMatched = new AtomicInteger(0);
+    private final String logPrefix;
+    private final AtomicBoolean cancelled = new AtomicBoolean(false);
+    private volatile AsyncConcurrentTaskExecutor<String> asyncConcurrentTaskExecutor;
+
+    @VisibleForTesting
+    protected LiveMigrationFilesVerificationTask(Builder builder)
+    {
+        this.vertx = builder.vertx;
+        this.id = builder.id;
+        this.source = builder.source;
+        this.port = builder.port;
+        this.instanceMetadata = builder.instanceMetadata;
+        this.executorPools = builder.executorPools;
+        this.request = builder.request;
+        this.sidecarClient = builder.sidecarClient;
+        this.liveMigrationConfiguration = builder.liveMigrationConfiguration;
+        this.digestVerifierFactory = builder.digestVerifierFactory;
+        this.logPrefix = "fileVerifyTaskId=" + id;
+        this.state = new AtomicReference<>(State.NOT_STARTED);
+    }
+
+    public static Builder builder()
+    {
+        return new Builder();
+    }
+
+    public Future<Void> validate()
+    {
+        return abortIfCancelled(new Object())
+               // Get files info from both source and destination
+               .compose(v -> Future.join(listFilesInLocal(), fetchSourceFileList()))
+
+               // Compare the files information before calculating digests
+               .compose(this::abortIfCancelled)
+               .compose(cf -> compareFilesMeta(cf.resultAt(0), cf.resultAt(1)))
+
+               // Create digest comparison tasks
+               .compose(this::abortIfCancelled)
+               .compose(this::getDigestComparisonTasks)
+
+               // Compare digests
+               .compose(this::abortIfCancelled)
+               .compose(this::compareDigests)
+
+               .andThen(ar -> {
+                   if (ar.failed())
+                   {
+                       LOGGER.error("{} Files verification task failed during validation", logPrefix, ar.cause());
+                       // Fail the task if it is in IN_PROGRESS state.
+                       state.compareAndSet(State.IN_PROGRESS, State.FAILED);
+                   }
+                   else
+                   {
+                       State currentState = state.compareAndExchange(State.IN_PROGRESS, State.COMPLETED);
+                       if (currentState == State.CANCELLED)
+                       {
+                           LOGGER.debug("{} Task was cancelled before completion", logPrefix);
+                       }
+                       else if (currentState != State.IN_PROGRESS)
+                       {
+                           LOGGER.warn("{} Unexpected state transition failure. State was {}", logPrefix, currentState);
+                       }
+                       else
+                       {
+                           LOGGER.info("{} Files verification task completed successfully", logPrefix);
+                       }
+                   }
+               });
+    }
+
+    private Future<List<InstanceFileInfo>> listFilesInLocal()
+    {
+        return executorPools.internal()
+                            .executeBlocking(() -> new CassandraInstanceFilesImpl(instanceMetadata,
+                                                                                  liveMigrationConfiguration)
+                                                   .files());
+    }
+
+    private Future<InstanceFilesListResponse> fetchSourceFileList()
+    {
+        return Future.fromCompletionStage(
+                     sidecarClient.liveMigrationListInstanceFilesAsync(new SidecarInstanceImpl(source, port)))
+                     .onFailure(cause -> LOGGER.error("{} Failed to obtain list of files from source {}:{}",
+                                                      logPrefix, source, port, cause));
+    }
+
+    private @NotNull Future<List<InstanceFileInfo>> compareFilesMeta(List<InstanceFileInfo> localFiles,
+                                                                     InstanceFilesListResponse sourceFiles)
+    {
+        Map<String, InstanceFileInfo> filesAtLocal =
+        localFiles.stream()
+                  .collect(Collectors.toMap(fileInfo -> fileInfo.fileUrl, fileInfo -> fileInfo));
+
+        Map<String, InstanceFileInfo> filesAtSource =
+        sourceFiles.getFiles().stream()
+                   .collect(Collectors.toMap(fileInfo -> fileInfo.fileUrl, fileInfo -> fileInfo));
+
+        for (Map.Entry<String, InstanceFileInfo> localFileEntry : filesAtLocal.entrySet())
+        {
+            String fileUrl = localFileEntry.getKey();
+            InstanceFileInfo localFile = localFileEntry.getValue();
+            if (filesAtSource.containsKey(fileUrl))
+            {
+                InstanceFileInfo sourceFile = filesAtSource.get(fileUrl);
+                // compare files metadata
+                if (localFile.equals(sourceFile) ||
+                    (localFile.fileType == DIRECTORY && sourceFile.fileType == DIRECTORY))
+                {
+                    // File info matches if either of them are directories or
+                    // their size, last modified time etc... matches
+                    LOGGER.debug("{} {} file metadata matched with source", logPrefix, fileUrl);
+                    metadataMatched.incrementAndGet();
+                }
+                else
+                {
+                    // Files metadata did not match
+                    metadataMismatches.incrementAndGet();
+                    LOGGER.error("{} {} did not match with source. Local file info={}, source file info={}",
+                                 logPrefix, fileUrl, localFile, sourceFile);
+                }
+
+                // done with processing current local file, remove it
+                filesAtSource.remove(fileUrl);
+            }
+            else
+            {
+                // File is not available at source, report it
+                LOGGER.error("{} File {} exists at destination but not found at source", logPrefix, fileUrl);
+                filesNotFoundAtSource.incrementAndGet();
+            }
+        }
+
+        for (Map.Entry<String, InstanceFileInfo> sourceFileEntry : filesAtSource.entrySet())
+        {
+            // Remaining files are not present at destination, report error for them
+            LOGGER.error("{} File {} exists at source but not found at destination", logPrefix, sourceFileEntry.getKey());
+            filesNotFoundAtDestination.incrementAndGet();
+        }
+
+        if (filesNotFoundAtDestination.get() > 0
+            || filesNotFoundAtSource.get() > 0
+            || metadataMismatches.get() > 0)
+        {
+            FileVerificationFailureException exception =
+            new FileVerificationFailureException("Files list did not match between source and destination");
+
+            return Future.failedFuture(exception);
+        }
+
+        return Future.succeededFuture(localFiles);
+    }
+
+    private @NotNull Future<List<Future<String>>> getDigestComparisonTasks(List<InstanceFileInfo> localFiles)
+    {
+        // now verify digests of each local file with source file
+        List<Callable<Future<String>>> tasks = new ArrayList<>(localFiles.size());
+        localFiles.stream()
+                  .filter(fileInfo -> fileInfo.fileType != DIRECTORY)
+                  .forEach(fileInfo -> tasks.add(() -> verifyDigest(fileInfo)));
+        asyncConcurrentTaskExecutor = new AsyncConcurrentTaskExecutor<>(vertx, tasks, request.maxConcurrency());
+        List<Future<String>> verifyTaskResults = asyncConcurrentTaskExecutor.start();
+        return Future.join(verifyTaskResults)
+                     .transform(ar -> Future.succeededFuture(verifyTaskResults));
+    }
+
+    private Future<Void> compareDigests(List<Future<String>> futures)
+    {
+        boolean verificationFailed = false;
+        for (Future<String> future : futures)
+        {
+            if (future.failed())
+            {
+                verificationFailed = true;
+                Throwable cause = future.cause();
+                if (cause instanceof DigestMismatchException)
+                {
+                    DigestMismatchException ex = (DigestMismatchException) cause;
+                    LOGGER.error("{} File digests did not match for {}.", logPrefix, ex.fileUrl(), cause);
+                    digestMismatches.incrementAndGet();
+                }
+                else
+                {
+                    LOGGER.error("{} Failed to verify digest for file. Check previous logs for file details",
+                                 logPrefix, cause);
+                    digestVerificationFailures.incrementAndGet();
+                }
+                continue;
+            }
+
+            // Digest verification succeeded
+            filesMatched.incrementAndGet();
+        }
+
+        if (verificationFailed)
+        {
+            return Future.failedFuture(
+            new FileVerificationFailureException("File digests did not match between source and destination"));
+        }
+        else
+        {
+            return Future.succeededFuture();
+        }
+    }
+
+    @VisibleForTesting
+    Future<String> verifyDigest(InstanceFileInfo fileInfo)
+    {
+        return getSourceFileDigest(fileInfo)
+               .compose(digest -> {
+                   String path = localPath(fileInfo.fileUrl, instanceMetadata).toAbsolutePath().toString();
+                   return digestVerifierFactory.verifier(MultiMap.caseInsensitiveMultiMap().addAll(digest.headers()))
+                                               .verify(path)
+                                               .compose(verified -> Future.succeededFuture(path))
+                                               .recover(cause -> Future.failedFuture(
+                                               new DigestMismatchException(path, fileInfo.fileUrl, cause)));
+               })
+               .onSuccess(filePath -> LOGGER.debug("{} Verified file {}", logPrefix, fileInfo.fileUrl))
+               .onFailure(cause -> LOGGER.error("{} Failed to verify file {}", logPrefix, fileInfo.fileUrl, cause));
+    }
+
+    private Future<Digest> getSourceFileDigest(InstanceFileInfo fileInfo)
+    {
+        return Future.fromCompletionStage(sidecarClient.liveMigrationFileDigestAsync(new SidecarInstanceImpl(source, port),
+                                                                                     fileInfo.fileUrl,
+                                                                                     request.digestAlgorithm()))
+                     .compose(this::toDigest);
+    }

Review Comment:
   It makes 1 http request to source per file to get the digest. According to `LiveMigrationConcurrencyLimitHandler`, `TOO_MANY_REQUESTS` can be thrown. There is no retry implemented to handle it, due to `SingleInstanceSelectionPolicy` + default retry policy. I think you want to add custom retry policy for the applicable requests. 
   
   Beside no retry and fail silently, 1 request per file seems to ensure slowness already. Maybe we should revisit the design decision later. 



##########
server/src/main/java/org/apache/cassandra/sidecar/livemigration/FilesVerificationTaskManager.java:
##########
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.sidecar.livemigration;
+
+import java.util.List;
+import java.util.Objects;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.datastax.driver.core.utils.UUIDs;
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import io.vertx.core.Future;
+import org.apache.cassandra.sidecar.cluster.instance.InstanceMetadata;
+import org.apache.cassandra.sidecar.common.request.LiveMigrationFilesVerificationRequest;
+import org.apache.cassandra.sidecar.common.response.LiveMigrationFilesVerificationResponse;
+import org.apache.cassandra.sidecar.config.SidecarConfiguration;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationInvalidRequestException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskInProgressException;
+import org.apache.cassandra.sidecar.exceptions.LiveMigrationExceptions.LiveMigrationTaskNotFoundException;
+import org.jetbrains.annotations.NotNull;
+
+import static org.apache.cassandra.sidecar.livemigration.LiveMigrationFilesVerificationTask.FILES_VERIFICATION_TASK_TYPE;
+
+/**
+ * Manages the lifecycle of file digest verification tasks during live migration operations.
+ * This manager ensures that only one {@link LiveMigrationTask} can be active per instance at a time,
+ * preventing concurrent migration operations that could impact system resources.
+ * Tasks are created using the {@link LiveMigrationFilesVerificationTaskFactory} and
+ * executed asynchronously to validate file integrity between source and destination nodes.
+ */
+@Singleton
+public class FilesVerificationTaskManager

Review Comment:
   Should `FilesVerificationTaskManager` and `DataCopyTaskManager` have a common base class? There are several almost identical methods, e.g. `getAllTasks()`, `getTask()` and `cancelTask()`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org