You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "denis-chudov (via GitHub)" <gi...@apache.org> on 2024/09/04 17:51:14 UTC

[PR] IGNITE-23125 Fix AssertionError: Unexpected primary replica state STOPPED on partition restart [ignite-3]

denis-chudov opened a new pull request, #4329:
URL: https://github.com/apache/ignite-3/pull/4329

   https://issues.apache.org/jira/browse/IGNITE-23125


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23125 Fix AssertionError: Unexpected primary replica state STOPPED on partition restart [ignite-3]

Posted by "vldpyatkov (via GitHub)" <gi...@apache.org>.
vldpyatkov commented on code in PR #4329:
URL: https://github.com/apache/ignite-3/pull/4329#discussion_r1755271064


##########
modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/LeaseUpdater.java:
##########
@@ -370,38 +397,32 @@ private void updateLeaseBatchInternal() {
                 if (!lease.isAccepted()) {
                     LeaseAgreement agreement = leaseNegotiator.getAndRemoveIfReady(grpId);
 
-                    agreement.checkValid(grpId, topologyTracker.currentTopologySnapshot(), assignments);
+                    if (lease.isProlongable()) {
+                        agreement.checkValid(grpId, topologyTracker.currentTopologySnapshot(), assignments);
 
-                    if (agreement.isAccepted()) {
-                        Lease negotiatedLease = agreement.getLease();
+                        if (agreement.isAccepted()) {
+                            Lease negotiatedLease = agreement.getLease();
 
-                        // Lease information is taken from lease tracker, where it appears on meta storage watch updates, so it can contain
-                        // stale leases, if watch processing was delayed for some reason. It is ok: negotiated lease is guaranteed to be
-                        // already written to meta storage before negotiation begins, and in this case its start time would be
-                        // greater than lease's.
-                        assert negotiatedLease.getStartTime().longValue() >= lease.getStartTime().longValue()
-                                : format("Can't publish the lease that was not negotiated [groupId={}, startTime={}, "
+                            // Lease information is taken from lease tracker, where it appears on meta storage watch updates, so it can
+                            // contain stale leases, if watch processing was delayed for some reason. It is ok: negotiated lease is
+                            // guaranteed to be already written to meta storage before negotiation begins, and in this case its start time
+                            // would be greater than lease's.
+                            assert negotiatedLease.getStartTime().longValue() >= lease.getStartTime().longValue()
+                                    : format("Can't publish the lease that was not negotiated [groupId={}, startTime={}, "
                                     + "agreementLeaseStartTime={}].", grpId, lease.getStartTime(), agreement.getLease().getStartTime());
 
-                        publishLease(grpId, negotiatedLease, renewedLeases);
-
-                        continue;
-                    } else if (agreement.isDeclined()) {
-                        // Here we initiate negotiations for UNDEFINED_AGREEMENT and retry them on newly started active actor as well.
-                        ClusterNode candidate = nextLeaseHolder(assignments, grpId, agreement.getRedirectTo());
+                            publishLease(grpId, negotiatedLease, renewedLeases);
 
-                        if (candidate == null) {
-                            leaseUpdateStatistics.onLeaseWithoutCandidate();
+                            continue;
+                        } else if (agreement.isDeclined()) {
+                            // Here we initiate negotiations for UNDEFINED_AGREEMENT and retry them on newly started active actor as well.
+                            chooseCandidateAndCreateNewLease(grpId, lease, agreement, assignments, renewedLeases, toBeNegotiated);
 
                             continue;
                         }
-
-                        // New lease is granted.
-                        writeNewLease(grpId, candidate, renewedLeases);
-
-                        boolean force = Objects.equals(lease.getLeaseholder(), candidate.name());
-
-                        toBeNegotiated.put(grpId, force);
+                    } else {
+                        // If the lease was denied, create the new one.
+                        chooseCandidateAndCreateNewLease(grpId, lease, agreement, assignments, renewedLeases, toBeNegotiated);

Review Comment:
   Here we have the same code in two condition's branches. You can make it once.



##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -546,23 +554,76 @@ private void onPlacementDriverMessageReceived(NetworkMessage msg0, ClusterNode s
      *
      * @param groupId Replication group id.
      * @param redirectNodeId Node consistent id to redirect.
+     * @return Future that is completed when the lease is denied to prolong, containing the expiration time of this lease.
      */
-    private void stopLeaseProlongation(ReplicationGroupId groupId, @Nullable String redirectNodeId) {
-        LOG.info("The replica does not meet the requirements for the leaseholder [groupId={}, redirectNodeId={}]", groupId, redirectNodeId);
-
-        msNodes.thenAccept(nodeIds -> {
-            for (String nodeId : nodeIds) {
-                ClusterNode node = clusterNetSvc.topologyService().getByConsistentId(nodeId);
-
-                if (node != null) {
-                    // TODO: IGNITE-19441 Stop lease prolongation message might be sent several
-                    clusterNetSvc.messagingService().send(node, PLACEMENT_DRIVER_MESSAGES_FACTORY.stopLeaseProlongationMessage()
-                            .groupId(groupId)
-                            .redirectProposal(redirectNodeId)
-                            .build());
+    private CompletableFuture<HybridTimestamp> stopLeaseProlongation(
+            ReplicationGroupId groupId,
+            @Nullable String redirectNodeId
+    ) {
+        long retriesTimeout = 60_000;

Review Comment:
   Make it static.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23125 Fix AssertionError: Unexpected primary replica state STOPPED on partition restart [ignite-3]

Posted by "denis-chudov (via GitHub)" <gi...@apache.org>.
denis-chudov commented on code in PR #4329:
URL: https://github.com/apache/ignite-3/pull/4329#discussion_r1755011204


##########
modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/LeaseUpdater.java:
##########
@@ -230,23 +253,60 @@ private CompletableFuture<Boolean> denyLease(ReplicationGroupId grpId, Lease lea
 
         Collection<Lease> leases = leasesCurrent.leaseByGroupId().values();
 
+        return denyLeaseImMetaStorage(deniedLease, leases, leasesCurrent.leasesBytes());
+    }
+
+    private CompletableFuture<Boolean> denyLeaseImMetaStorage(
+            Lease deniedLease,
+            Collection<Lease> currentLeases,
+            byte[] currentLeasesBytes
+    ) {
+        ByteArray key = PLACEMENTDRIVER_LEASES_KEY;
+
+        IgniteBiTuple<List<Lease>, Boolean> renewedLeasesTup = replaceLeaseInCollection(currentLeases, deniedLease);
+
+        if (!renewedLeasesTup.get2()) {
+            return falseCompletedFuture();
+        } else {
+            return msManager.invoke(
+                    or(notExists(key), value(key).eq(currentLeasesBytes)),
+                    put(key, new LeaseBatch(renewedLeasesTup.get1()).bytes()),
+                    noop()
+            ).thenCompose(res -> {
+                if (res) {
+                    return trueCompletedFuture();
+                } else {
+                    return refreshLeasesFromMetaStorage()

Review Comment:
   reworked to client side retries



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23125 Fix AssertionError: Unexpected primary replica state STOPPED on partition restart [ignite-3]

Posted by "denis-chudov (via GitHub)" <gi...@apache.org>.
denis-chudov commented on code in PR #4329:
URL: https://github.com/apache/ignite-3/pull/4329#discussion_r1755009189


##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -548,21 +552,39 @@ private void onPlacementDriverMessageReceived(NetworkMessage msg0, ClusterNode s
      * @param redirectNodeId Node consistent id to redirect.
      */
     private void stopLeaseProlongation(ReplicationGroupId groupId, @Nullable String redirectNodeId) {
-        LOG.info("The replica does not meet the requirements for the leaseholder [groupId={}, redirectNodeId={}]", groupId, redirectNodeId);
+        stopLeaseProlongation(groupId, redirectNodeId, false);
+    }
+
+    /**
+     * Sends stop lease prolongation message to all participants of placement driver group.
+     *
+     * @param groupId Replication group id.
+     * @param redirectNodeId Node consistent id to redirect.
+     * @param waitForPrimary Whether wait for primary to appear.
+     * @return Future that is completed when {@link StopLeaseProlongationMessage} is sent.
+     */
+    private CompletableFuture<Void> stopLeaseProlongation(
+            ReplicationGroupId groupId,
+            @Nullable String redirectNodeId,
+            boolean waitForPrimary
+    ) {
+        CompletableFuture<ReplicaMeta> primaryReplicaFuture = waitForPrimary
+                ? placementDriver.awaitPrimaryReplica(groupId, clockService.now(), 120, TimeUnit.SECONDS)

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23125 Fix AssertionError: Unexpected primary replica state STOPPED on partition restart [ignite-3]

Posted by "vldpyatkov (via GitHub)" <gi...@apache.org>.
vldpyatkov merged PR #4329:
URL: https://github.com/apache/ignite-3/pull/4329


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23125 Fix AssertionError: Unexpected primary replica state STOPPED on partition restart [ignite-3]

Posted by "vldpyatkov (via GitHub)" <gi...@apache.org>.
vldpyatkov commented on code in PR #4329:
URL: https://github.com/apache/ignite-3/pull/4329#discussion_r1755219404


##########
modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/leases/Lease.java:
##########
@@ -153,10 +151,10 @@ public Lease acceptLease(HybridTimestamp to) {
      *
      * @return Denied lease.
      */
-    public Lease denyLease(String proposedCandidate) {
-        assert accepted : "The lease is not accepted: " + this;
+    public Lease denyLease(@Nullable String proposedCandidate) {
+        HybridTimestamp newExpirationTime = accepted ? expirationTime : new HybridTimestamp(System.currentTimeMillis(), 0);
 
-        return new Lease(leaseholder, leaseholderId, startTime, expirationTime, false, true, proposedCandidate, replicationGroupId);
+        return new Lease(leaseholder, leaseholderId, startTime, newExpirationTime, false, accepted, proposedCandidate, replicationGroupId);

Review Comment:
   The accepted flag should be set to true.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23125 Fix AssertionError: Unexpected primary replica state STOPPED on partition restart [ignite-3]

Posted by "denis-chudov (via GitHub)" <gi...@apache.org>.
denis-chudov commented on code in PR #4329:
URL: https://github.com/apache/ignite-3/pull/4329#discussion_r1756565974


##########
modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/LeaseUpdater.java:
##########
@@ -370,38 +397,32 @@ private void updateLeaseBatchInternal() {
                 if (!lease.isAccepted()) {
                     LeaseAgreement agreement = leaseNegotiator.getAndRemoveIfReady(grpId);
 
-                    agreement.checkValid(grpId, topologyTracker.currentTopologySnapshot(), assignments);
+                    if (lease.isProlongable()) {
+                        agreement.checkValid(grpId, topologyTracker.currentTopologySnapshot(), assignments);
 
-                    if (agreement.isAccepted()) {
-                        Lease negotiatedLease = agreement.getLease();
+                        if (agreement.isAccepted()) {
+                            Lease negotiatedLease = agreement.getLease();
 
-                        // Lease information is taken from lease tracker, where it appears on meta storage watch updates, so it can contain
-                        // stale leases, if watch processing was delayed for some reason. It is ok: negotiated lease is guaranteed to be
-                        // already written to meta storage before negotiation begins, and in this case its start time would be
-                        // greater than lease's.
-                        assert negotiatedLease.getStartTime().longValue() >= lease.getStartTime().longValue()
-                                : format("Can't publish the lease that was not negotiated [groupId={}, startTime={}, "
+                            // Lease information is taken from lease tracker, where it appears on meta storage watch updates, so it can
+                            // contain stale leases, if watch processing was delayed for some reason. It is ok: negotiated lease is
+                            // guaranteed to be already written to meta storage before negotiation begins, and in this case its start time
+                            // would be greater than lease's.
+                            assert negotiatedLease.getStartTime().longValue() >= lease.getStartTime().longValue()
+                                    : format("Can't publish the lease that was not negotiated [groupId={}, startTime={}, "
                                     + "agreementLeaseStartTime={}].", grpId, lease.getStartTime(), agreement.getLease().getStartTime());
 
-                        publishLease(grpId, negotiatedLease, renewedLeases);
-
-                        continue;
-                    } else if (agreement.isDeclined()) {
-                        // Here we initiate negotiations for UNDEFINED_AGREEMENT and retry them on newly started active actor as well.
-                        ClusterNode candidate = nextLeaseHolder(assignments, grpId, agreement.getRedirectTo());
+                            publishLease(grpId, negotiatedLease, renewedLeases);
 
-                        if (candidate == null) {
-                            leaseUpdateStatistics.onLeaseWithoutCandidate();
+                            continue;
+                        } else if (agreement.isDeclined()) {
+                            // Here we initiate negotiations for UNDEFINED_AGREEMENT and retry them on newly started active actor as well.
+                            chooseCandidateAndCreateNewLease(grpId, lease, agreement, assignments, renewedLeases, toBeNegotiated);
 
                             continue;
                         }
-
-                        // New lease is granted.
-                        writeNewLease(grpId, candidate, renewedLeases);
-
-                        boolean force = Objects.equals(lease.getLeaseholder(), candidate.name());
-
-                        toBeNegotiated.put(grpId, force);
+                    } else {
+                        // If the lease was denied, create the new one.
+                        chooseCandidateAndCreateNewLease(grpId, lease, agreement, assignments, renewedLeases, toBeNegotiated);

Review Comment:
   refactored



##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -546,23 +554,76 @@ private void onPlacementDriverMessageReceived(NetworkMessage msg0, ClusterNode s
      *
      * @param groupId Replication group id.
      * @param redirectNodeId Node consistent id to redirect.
+     * @return Future that is completed when the lease is denied to prolong, containing the expiration time of this lease.
      */
-    private void stopLeaseProlongation(ReplicationGroupId groupId, @Nullable String redirectNodeId) {
-        LOG.info("The replica does not meet the requirements for the leaseholder [groupId={}, redirectNodeId={}]", groupId, redirectNodeId);
-
-        msNodes.thenAccept(nodeIds -> {
-            for (String nodeId : nodeIds) {
-                ClusterNode node = clusterNetSvc.topologyService().getByConsistentId(nodeId);
-
-                if (node != null) {
-                    // TODO: IGNITE-19441 Stop lease prolongation message might be sent several
-                    clusterNetSvc.messagingService().send(node, PLACEMENT_DRIVER_MESSAGES_FACTORY.stopLeaseProlongationMessage()
-                            .groupId(groupId)
-                            .redirectProposal(redirectNodeId)
-                            .build());
+    private CompletableFuture<HybridTimestamp> stopLeaseProlongation(
+            ReplicationGroupId groupId,
+            @Nullable String redirectNodeId
+    ) {
+        long retriesTimeout = 60_000;

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23125 Fix AssertionError: Unexpected primary replica state STOPPED on partition restart [ignite-3]

Posted by "vldpyatkov (via GitHub)" <gi...@apache.org>.
vldpyatkov commented on code in PR #4329:
URL: https://github.com/apache/ignite-3/pull/4329#discussion_r1751899387


##########
modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/LeaseUpdater.java:
##########
@@ -230,23 +253,60 @@ private CompletableFuture<Boolean> denyLease(ReplicationGroupId grpId, Lease lea
 
         Collection<Lease> leases = leasesCurrent.leaseByGroupId().values();
 
+        return denyLeaseImMetaStorage(deniedLease, leases, leasesCurrent.leasesBytes());
+    }
+
+    private CompletableFuture<Boolean> denyLeaseImMetaStorage(
+            Lease deniedLease,
+            Collection<Lease> currentLeases,
+            byte[] currentLeasesBytes
+    ) {
+        ByteArray key = PLACEMENTDRIVER_LEASES_KEY;
+
+        IgniteBiTuple<List<Lease>, Boolean> renewedLeasesTup = replaceLeaseInCollection(currentLeases, deniedLease);
+
+        if (!renewedLeasesTup.get2()) {
+            return falseCompletedFuture();
+        } else {
+            return msManager.invoke(
+                    or(notExists(key), value(key).eq(currentLeasesBytes)),
+                    put(key, new LeaseBatch(renewedLeasesTup.get1()).bytes()),
+                    noop()
+            ).thenCompose(res -> {
+                if (res) {
+                    return trueCompletedFuture();
+                } else {
+                    return refreshLeasesFromMetaStorage()

Review Comment:
   I am concerned about thread starvation in this approach. Probably we can return to the base one and won't retry invoke on the server side.



##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -548,21 +552,39 @@ private void onPlacementDriverMessageReceived(NetworkMessage msg0, ClusterNode s
      * @param redirectNodeId Node consistent id to redirect.
      */
     private void stopLeaseProlongation(ReplicationGroupId groupId, @Nullable String redirectNodeId) {
-        LOG.info("The replica does not meet the requirements for the leaseholder [groupId={}, redirectNodeId={}]", groupId, redirectNodeId);
+        stopLeaseProlongation(groupId, redirectNodeId, false);
+    }
+
+    /**
+     * Sends stop lease prolongation message to all participants of placement driver group.
+     *
+     * @param groupId Replication group id.
+     * @param redirectNodeId Node consistent id to redirect.
+     * @param waitForPrimary Whether wait for primary to appear.
+     * @return Future that is completed when {@link StopLeaseProlongationMessage} is sent.
+     */
+    private CompletableFuture<Void> stopLeaseProlongation(
+            ReplicationGroupId groupId,
+            @Nullable String redirectNodeId,
+            boolean waitForPrimary
+    ) {
+        CompletableFuture<ReplicaMeta> primaryReplicaFuture = waitForPrimary
+                ? placementDriver.awaitPrimaryReplica(groupId, clockService.now(), 120, TimeUnit.SECONDS)

Review Comment:
   How do you identify a timeout to wait primary here?
   After a short discussion, we decided to avoid waiting primary here. Instead of this, we will send the stop lease prolangation messaeg at any time, even if the node is not a primary in MC.



##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -548,21 +552,39 @@ private void onPlacementDriverMessageReceived(NetworkMessage msg0, ClusterNode s
      * @param redirectNodeId Node consistent id to redirect.
      */
     private void stopLeaseProlongation(ReplicationGroupId groupId, @Nullable String redirectNodeId) {
-        LOG.info("The replica does not meet the requirements for the leaseholder [groupId={}, redirectNodeId={}]", groupId, redirectNodeId);
+        stopLeaseProlongation(groupId, redirectNodeId, false);
+    }
+
+    /**
+     * Sends stop lease prolongation message to all participants of placement driver group.
+     *
+     * @param groupId Replication group id.
+     * @param redirectNodeId Node consistent id to redirect.
+     * @param waitForPrimary Whether wait for primary to appear.
+     * @return Future that is completed when {@link StopLeaseProlongationMessage} is sent.
+     */
+    private CompletableFuture<Void> stopLeaseProlongation(
+            ReplicationGroupId groupId,
+            @Nullable String redirectNodeId,
+            boolean waitForPrimary
+    ) {
+        CompletableFuture<ReplicaMeta> primaryReplicaFuture = waitForPrimary
+                ? placementDriver.awaitPrimaryReplica(groupId, clockService.now(), 120, TimeUnit.SECONDS)

Review Comment:
   Additionally, we are going to improve the protocol for stopping lease prolongation. In the proposal, the server (PD) sends a response with the right border for the current lease. This border allows the node to remove primary reservations from the replica.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23125 Fix AssertionError: Unexpected primary replica state STOPPED on partition restart [ignite-3]

Posted by "denis-chudov (via GitHub)" <gi...@apache.org>.
denis-chudov commented on code in PR #4329:
URL: https://github.com/apache/ignite-3/pull/4329#discussion_r1755275285


##########
modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/leases/Lease.java:
##########
@@ -153,10 +151,10 @@ public Lease acceptLease(HybridTimestamp to) {
      *
      * @return Denied lease.
      */
-    public Lease denyLease(String proposedCandidate) {
-        assert accepted : "The lease is not accepted: " + this;
+    public Lease denyLease(@Nullable String proposedCandidate) {
+        HybridTimestamp newExpirationTime = accepted ? expirationTime : new HybridTimestamp(System.currentTimeMillis(), 0);
 
-        return new Lease(leaseholder, leaseholderId, startTime, expirationTime, false, true, proposedCandidate, replicationGroupId);
+        return new Lease(leaseholder, leaseholderId, startTime, newExpirationTime, false, accepted, proposedCandidate, replicationGroupId);

Review Comment:
   We can't do this: replica must accept the lease and send LeaseGrantMessageResponse before that. Otherwise we can grant the accepted lease to replica that could have not caught up yet.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23125 Fix AssertionError: Unexpected primary replica state STOPPED on partition restart [ignite-3]

Posted by "denis-chudov (via GitHub)" <gi...@apache.org>.
denis-chudov commented on code in PR #4329:
URL: https://github.com/apache/ignite-3/pull/4329#discussion_r1755275285


##########
modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/leases/Lease.java:
##########
@@ -153,10 +151,10 @@ public Lease acceptLease(HybridTimestamp to) {
      *
      * @return Denied lease.
      */
-    public Lease denyLease(String proposedCandidate) {
-        assert accepted : "The lease is not accepted: " + this;
+    public Lease denyLease(@Nullable String proposedCandidate) {
+        HybridTimestamp newExpirationTime = accepted ? expirationTime : new HybridTimestamp(System.currentTimeMillis(), 0);
 
-        return new Lease(leaseholder, leaseholderId, startTime, expirationTime, false, true, proposedCandidate, replicationGroupId);
+        return new Lease(leaseholder, leaseholderId, startTime, newExpirationTime, false, accepted, proposedCandidate, replicationGroupId);

Review Comment:
   We can't do this: replica must accept the lease and send LeaseGrantMessageResponse before that. Otherwise we can grant the accepted lease to replica that could not have caught up yet.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org